Assigning geometry for pretesting against screen regions for an image frame using prior frame information

ABSTRACT

A method including rendering graphics for an application using graphics processing units (GPUs). Responsibility for rendering of geometry is divided between GPUs based on screen regions, each GPU having a corresponding division of the responsibility which is known. First pieces of geometry are rendered at the GPUs during a rendering phase of a previous image frame. Statistics are generated for the rendering of the previous image frame. Second pieces of geometry of a current image frame are assigned based on the statistics to the GPUs for geometry testing. Geometry testing at a current image frame on the second pieces of geometry is performed to generate information regarding each piece of geometry and its relation to each screen region, the geometry testing performed at each of the GPUs based on the assigning. The information generated for the second pieces of geometry is used when rendering the geometry at the GPUs.

CLAIM OF PRIORITY

This application is a continuation of and claims priority to and thebenefit of commonly owned, patent application U.S. Ser. No. 17/389,311,filed on Jul. 29, 2021, Attorney Docket No. SONYP426B.C1, entitled“System and Method for Efficient Multi-GPU Rendering of Geometry byPretesting Against Screen Regions Using Prior Frame Information”; whichis a continuation of and claims priority to and the benefit of commonlyowned, patent application U.S. Ser. No. 16/780,722, filed on Feb. 3,2020, Attorney Docket No. SONYP426B, entitled “System and Method forEfficient Multi-GPU Rendering of Geometry by Pretesting Against ScreenRegions Using Prior Frame Information”; the disclosures of which areincorporated herein in their entireties for all purposes.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to commonly assigned, co-pending U.S. patentapplication Ser. No. 16/780,680, entitled “SYSTEM AND METHOD FOREFFICIENT MULTI-GPU RENDERING OF GEOMETRY BY PRETESTING AGAINSTINTERLEAVED SCREEN REGIONS BEFORE RENDERING,” Attorney Docket No.SONYP426A, filed concurrently with the present application, thedisclosure of which is hereby incorporated by reference in its entirety.This application is related to commonly assigned, co-pending U.S. patentapplication Ser. No. 16/780,745, entitled “SYSTEM AND METHOD FOREFFICIENT MULTI-GPU RENDERING OF GEOMETRY BY PRETESTING AGAINST SCREENREGIONS USING CONFIGURABLE SHADERS,” Attorney Docket No. SONYP426C,filed concurrently with the present application, the disclosure of whichis hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure is related to graphic processing, and morespecifically for multi-GPU collaboration when rendering an image for anapplication.

BACKGROUND OF THE DISCLOSURE

In recent years there has been a continual push for online services thatallow for online or cloud gaming in a streaming format between a cloudgaming server and a client connected through a network. The streamingformat has increasingly become more popular because of the availabilityof game titles on demand, the ability to execute more complex games, theability to network between players for multi-player gaming, sharing ofassets between players, sharing of instant experiences between playersand/or spectators, allowing friends to watch a friend play a video game,having a friend join the on-going game play of a friend, and the like.

The cloud gaming server may be configured to provide resources to one ormore clients and/or applications. That is, the cloud gaming server maybe configured with resources capable of high throughput. For example,there are limits to the performance that an individual graphicsprocessing unit (GPU) can attain. To render even more complex scenes oruse even more complex algorithms (e.g. materials, lighting, etc.) whengenerating a scene, it may be desirable to use multiple GPUs to render asingle image. However, usage of those graphics processing units equallyis difficult to achieve. Further, even though there are multiple GPUs toprocess an image for an application using traditional technologies,there is not the ability to support a corresponding increase in bothscreen pixel count and density of geometry (e.g., four GPUs cannot writefour times the pixels and/or process four times the vertices orprimitives for an image).

It is in this context that embodiments of the disclosure arise.

SUMMARY

Embodiments of the present disclosure relate to using multiple GPUs incollaboration to render a single image, such as multi-GPU rendering ofgeometry for an application by pretesting against screen regions, whichmay be interleaved, before rendering.

Embodiments of the present disclosure disclose a method for graphicsprocessing. The method includes rendering graphics for an applicationusing a plurality of graphics processing units (GPUs). The methodincludes dividing responsibility for rendering of geometry of thegraphics between the plurality of GPUs based on a plurality of screenregions, each GPU having a corresponding division of the responsibilitywhich is known to the plurality of GPUs. Screen regions are interleaved.The method includes assigning a plurality of pieces of geometry of animage frame to the plurality of GPUs for geometry testing. The methodincludes assigning a GPU a piece of geometry of an image frame generatedby an application for geometry testing. The method includes performinggeometry testing at the GPU to generate information regarding the pieceof geometry and its relation to each of the plurality of screen regions.The method includes rendering the piece of geometry using theinformation at each of the plurality of GPUs, where using theinformation can include, for example, skipping rendering entirely if ithas been determined that the piece of geometry does not overlap anyscreen region assigned to a given GPU.

In another embodiment, a non-transitory computer-readable medium forperforming a method is disclosed. The computer-readable medium includingprogram instructions for rendering graphics for an application using aplurality of graphics processing units (GPUs). The computer-readablemedium including program instructions for dividing responsibility forthe rendering of geometry of the graphics between the plurality of GPUsbased on a plurality of screen regions, each GPU having a correspondingdivision of the responsibility which is known to the plurality of GPUs,wherein screen regions in the plurality of screen regions areinterleaved. The computer readable medium including program instructionsfor assigning a GPU a piece of geometry of an image frame generated byan application for geometry pretesting. The computer readable mediumincluding program instructions for performing the geometry pretesting atthe GPU to generate information regarding the piece of geometry and itsrelation to each of the plurality of screen regions. The computerreadable medium including program instructions for using the informationat each of the plurality of GPUs when rendering the image frame.

In still another embodiment, a computer system is disclosed including aprocessor and memory coupled to the processor and having stored thereininstructions that, if executed by the computer system, cause thecomputer system to execute a method for graphics processing. The methodincluding rendering graphics for an application using a plurality ofgraphics processing units (GPUs). The method including dividingresponsibility for the rendering of geometry of the graphics between theplurality of GPUs based on a plurality of screen regions, each GPUhaving a corresponding division of the responsibility which is known tothe plurality of GPUs, wherein screen regions in the plurality of screenregions are interleaved. The method including assigning a GPU a piece ofgeometry of an image frame generated by an application for geometrypretesting. The method including performing the geometry pretesting atthe GPU to generate information regarding the piece of geometry and itsrelation to each of the plurality of screen regions. The methodincluding using the information at each of the plurality of GPUs whenrendering the image frame.

Embodiments of the present disclosure disclose a method for graphicsprocessing. The method includes rendering graphics for an applicationusing a plurality of graphics processing units (GPUs). The methodincludes dividing responsibility for the rendering of geometry of thegraphics between the plurality of GPUs based on a plurality of screenregions, each GPU having a corresponding division of the responsibilitywhich is known to the plurality of GPUs. The method includes performinggeometry testing at a pretest GPU on a plurality of pieces of geometryof an image frame generated by an application to generate informationregarding each piece of geometry and its relation to each of theplurality of screen regions. The method includes rendering the pluralityof pieces of geometry at each of the plurality of GPUs using theinformation generated for each of the plurality of pieces of geometry,where using the information includes, for example, skipping renderingentirely if it has been determined that the piece of geometry does notoverlap any screen region assigned to a given GPU.

In another embodiment, a non-transitory computer-readable medium forperforming a method is disclosed. The computer-readable medium includingprogram instructions for rendering graphics for an application using aplurality of graphics processing units (GPUs). The computer-readablemedium including program instructions for dividing responsibility forthe rendering of geometry of the graphics between the plurality of GPUsbased on a plurality of screen regions, each GPU having a correspondingdivision of the responsibility which is known to the plurality of GPUs.The computer-readable medium including program instructions forperforming geometry testing at a pretest GPU on a plurality of pieces ofgeometry of an image frame generated by an application to generateinformation regarding each piece of geometry and its relation to each ofthe plurality of screen regions. The computer-readable medium includingprogram instructions for rendering the plurality of pieces of geometryat each of the plurality of GPUs using the information generated foreach of the plurality of pieces of geometry, where using the informationincludes, for example, skipping rendering entirely if it has beendetermined that the piece of geometry does not overlap any screen regionassigned to a given GPU.

In still another embodiment, a computer system is disclosed including aprocessor and memory coupled to the processor and having stored thereininstructions that, if executed by the computer system, cause thecomputer system to execute a method for graphics processing. The methodincluding rendering graphics for an application using a plurality ofgraphics processing units (GPUs). The method includes dividingresponsibility for the rendering of geometry of the graphics between theplurality of GPUs based on a plurality of screen regions, each GPUhaving a corresponding division of the responsibility which is known tothe plurality of GPUs. The method includes performing geometry testingat a pretest GPU on a plurality of pieces of geometry of an image framegenerated by an application to generate information regarding each pieceof geometry and its relation to each of the plurality of screen regions.The method includes rendering the plurality of pieces of geometry ateach of the plurality of GPUs using the information generated for eachof the plurality of pieces of geometry, where using the informationincludes, for example, skipping rendering entirely if it has beendetermined that the piece of geometry does not overlap any screen regionassigned to a given GPU.

Embodiments of the present disclosure disclose a method for graphicsprocessing. The method includes rendering graphics for an applicationusing a plurality of graphics processing units (GPUs). The methodincludes dividing responsibility for the rendering of geometry of thegraphics between the plurality of GPUs based on a plurality of screenregions, each GPU having a corresponding division of the responsibilitywhich is known to the plurality of GPUs. The method includes rendering afirst plurality of pieces of geometry at the plurality of GPUs during arendering phase of a previous image frame generated by an application.The method includes generating statistics for the rendering of theprevious image frame. The method includes assigning based on thestatistics a second plurality of pieces of geometry of a current imageframe generated by the application to the plurality of GPUs for geometrytesting. The method includes performing geometry testing at a currentimage frame on the second plurality of pieces of geometry to generateinformation regarding each piece of the second plurality of pieces ofgeometry and its relation to each of the plurality of screen regions,wherein the geometry testing is performed at each of the plurality ofGPUs based on the assigning. The method includes rendering the secondplurality of pieces of geometry at each of the plurality of GPUs usingthe information generated for each of the second plurality of pieces ofgeometry, where using the information can include, for example, skippingrendering entirely if it has been determined that the piece of geometrydoes not overlap any screen region assigned to a given GPU.

In another embodiment, a non-transitory computer-readable medium forperforming a method is disclosed. The computer-readable medium includingprogram instructions for rendering graphics for an application using aplurality of graphics processing units (GPUs). The computer-readablemedium including program instructions for dividing responsibility forthe rendering of geometry of the graphics between the plurality of GPUsbased on a plurality of screen regions, each GPU having a correspondingdivision of the responsibility which is known to the plurality of GPUs.The computer-readable medium including program instructions forrendering a first plurality of pieces of geometry at the plurality ofGPUs during a rendering phase of a previous image frame generated by anapplication. The computer-readable medium including program instructionsfor generating statistics for the rendering of the previous image frame.The computer-readable medium including program instructions forassigning based on the statistics a second plurality of pieces ofgeometry of a current image frame generated by the application to theplurality of GPUs for geometry testing. The computer-readable mediumincluding program instructions for performing geometry testing at acurrent image frame on the second plurality of pieces of geometry togenerate information regarding each piece of the second plurality ofpieces of geometry and its relation to each of the plurality of screenregions, wherein the geometry testing is performed at each of theplurality of GPUs based on the assigning. The computer-readable mediumincluding program instructions for rendering the second plurality ofpieces of geometry at each of the plurality of GPUs using theinformation generated for each of the second plurality of pieces ofgeometry, where using the information can include, for example, skippingrendering entirely if it has been determined that the piece of geometrydoes not overlap any screen region assigned to a given GPU.

In still another embodiment, a computer system is disclosed including aprocessor and memory coupled to the processor and having stored thereininstructions that, if executed by the computer system, cause thecomputer system to execute a method for graphics processing. The methodincludes rendering graphics for an application using a plurality ofgraphics processing units (GPUs). The method includes dividingresponsibility for the rendering of geometry of the graphics between theplurality of GPUs based on a plurality of screen regions, each GPUhaving a corresponding division of the responsibility which is known tothe plurality of GPUs. The method includes rendering a first pluralityof pieces of geometry at the plurality of GPUs during a rendering phaseof a previous image frame generated by an application. The methodincludes generating statistics for the rendering of the previous imageframe. The method includes assigning based on the statistics a secondplurality of pieces of geometry of a current image frame generated bythe application to the plurality of GPUs for geometry testing. Themethod includes performing geometry testing at a current image frame onthe second plurality of pieces of geometry to generate informationregarding each piece of the second plurality of pieces of geometry andits relation to each of the plurality of screen regions, wherein thegeometry testing is performed at each of the plurality of GPUs based onthe assigning. The method includes rendering the second plurality ofpieces of geometry at each of the plurality of GPUs using theinformation generated for each of the second plurality of pieces ofgeometry, where using the information can include, for example, skippingrendering entirely if it has been determined that the piece of geometrydoes not overlap any screen region assigned to a given GPU.

Embodiments of the present disclosure disclose a method for graphicsprocessing. The method includes rendering graphics for an applicationusing a plurality of graphics processing units (GPUs). The methodincludes dividing responsibility for the rendering of geometry of thegraphics between the plurality of GPUs based on a plurality of screenregions, each GPU having a corresponding division of the responsibilitywhich is known to the plurality of GPUs. The method includes assigning aplurality of pieces of geometry of an image frame to the plurality ofGPUs for geometry testing. The method includes setting a first stateconfiguring one or more shaders to perform the geometry testing. Themethod includes performing geometry testing at the plurality of GPUs onthe plurality of pieces of geometry to generate information regardingeach piece of geometry and its relation to each of the plurality ofscreen regions. The method includes setting a second state configuringthe one or more shaders to perform rendering. The method includesrendering the plurality of pieces of geometry at each of the pluralityof GPUs using the information generated for each of the plurality ofpieces of geometry, where using the information includes, for example,skipping rendering entirely if it has been determined that the piece ofgeometry does not overlap any screen region assigned to a given GPU.

In another embodiment, a non-transitory computer-readable medium forperforming a method is disclosed. The computer-readable medium includingprogram instructions for rendering graphics for an application using aplurality of graphics processing units (GPUs). The computer-readablemedium including program instructions for dividing responsibility forthe rendering of geometry of the graphics between the plurality of GPUsbased on a plurality of screen regions, each GPU having a correspondingdivision of the responsibility which is known to the plurality of GPUs.The computer-readable medium including program instructions forassigning a plurality of pieces of geometry of an image frame to theplurality of GPUs for geometry testing. The computer-readable mediumincluding program instructions for setting a first state configuring oneor more shaders to perform the geometry testing. The computer-readablemedium including program instructions for performing geometry testing atthe plurality of GPUs on the plurality of pieces of geometry to generateinformation regarding each piece of geometry and its relation to each ofthe plurality of screen regions. The computer-readable medium includingprogram instructions for setting a second state configuring the one ormore shaders to perform rendering. The computer-readable mediumincluding program instructions for rendering the plurality of pieces ofgeometry at each of the plurality of GPUs using the informationgenerated for each of the plurality of pieces of geometry, where usingthe information includes, for example, skipping rendering entirely if ithas been determined that the piece of geometry does not overlap anyscreen region assigned to a given GPU.

In still another embodiment, a computer system is disclosed including aprocessor and memory coupled to the processor and having stored thereininstructions that, if executed by the computer system, cause thecomputer system to execute a method for graphics processing. The methodincludes rendering graphics for an application using a plurality ofgraphics processing units (GPUs). The method includes dividingresponsibility for the rendering of geometry of the graphics between theplurality of GPUs based on a plurality of screen regions, each GPUhaving a corresponding division of the responsibility which is known tothe plurality of GPUs. The method includes assigning a plurality ofpieces of geometry of an image frame to the plurality of GPUs forgeometry testing. The method includes setting a first state configuringone or more shaders to perform the geometry testing. The method includesperforming geometry testing at the plurality of GPUs on the plurality ofpieces of geometry to generate information regarding each piece ofgeometry and its relation to each of the plurality of screen regions.The method includes setting a second state configuring the one or moreshaders to perform rendering. The method includes rendering theplurality of pieces of geometry at each of the plurality of GPUs usingthe information generated for each of the plurality of pieces ofgeometry, where using the information includes, for example, skippingrendering entirely if it has been determined that the piece of geometrydoes not overlap any screen region assigned to a given GPU.

Embodiments of the present disclosure disclose a method for graphicsprocessing. The method includes rendering graphics for an applicationusing a plurality of graphics processing units (GPUs). The methodincludes dividing responsibility for the rendering of geometry of thegraphics between the plurality of GPUs based on a plurality of screenregions, each GPU having a corresponding division of the responsibilitywhich is known to the plurality of GPUs. The method includes assigning aplurality of pieces of geometry of an image frame to the plurality ofGPUs for geometry testing. The method includes interleaving a first setof shaders to perform geometry testing and rendering on a first set ofpieces of geometry with a second set of shaders to perform geometrytesting and rendering on a second set of pieces of geometry. Thegeometry testing generates corresponding information regarding eachpiece of geometry in the first set or second set and its relation toeach of the plurality of screen regions. The corresponding informationis used by the plurality of GPUs to render each piece of geometry infirst set or second set, where using the information includes, forexample, skipping rendering entirely if it has been determined that thepiece of geometry does not overlap any screen region assigned to a givenGPU.

In another embodiment, a non-transitory computer-readable medium forperforming a method is disclosed. The computer-readable medium includingprogram instructions for rendering graphics for an application using aplurality of graphics processing units (GPUs). The computer-readablemedium including program instructions for dividing responsibility forthe rendering of geometry of the graphics between the plurality of GPUsbased on a plurality of screen regions, each GPU having a correspondingdivision of the responsibility which is known to the plurality of GPUs.The computer-readable medium including program instructions forassigning a plurality of pieces of geometry of an image frame to theplurality of GPUs for geometry testing. The computer-readable mediumincluding program instructions for interleaving a first set of shadersto perform geometry testing and rendering on a first set of pieces ofgeometry with a second set of shaders to perform geometry testing andrendering on a second set of pieces of geometry. The geometry testinggenerates corresponding information regarding each piece of geometry inthe first set or second set and its relation to each of the plurality ofscreen regions. The corresponding information is used by the pluralityof GPUs to render each piece of geometry in first set or second set,where using the information includes, for example, skipping renderingentirely if it has been determined that the piece of geometry does notoverlap any screen region assigned to a given GPU.

In still another embodiment, a computer system is disclosed including aprocessor and memory coupled to the processor and having stored thereininstructions that, if executed by the computer system, cause thecomputer system to execute a method for graphics processing. The methodincludes rendering graphics for an application using a plurality ofgraphics processing units (GPUs). The method includes dividingresponsibility for the rendering of geometry of the graphics between theplurality of GPUs based on a plurality of screen regions, each GPUhaving a corresponding division of the responsibility which is known tothe plurality of GPUs. The method includes assigning a plurality ofpieces of geometry of an image frame to the plurality of GPUs forgeometry testing. The method includes interleaving a first set ofshaders to perform geometry testing and rendering on a first set ofpieces of geometry with a second set of shaders to perform geometrytesting and rendering on a second set of pieces of geometry. Thegeometry testing generates corresponding information regarding eachpiece of geometry in the first set or second set and its relation toeach of the plurality of screen regions. The corresponding informationis used by the plurality of GPUs to render each piece of geometry infirst set or second set, where using the information includes, forexample, skipping rendering entirely if it has been determined that thepiece of geometry does not overlap any screen region assigned to a givenGPU.

Other aspects of the disclosure will become apparent from the followingdetailed description, taken in conjunction with the accompanyingdrawings, illustrating by way of example the principles of thedisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 is a diagram of a system for providing gaming over a networkbetween one or more cloud gaming servers configured for implementingmultiple GPUs in collaboration to render a single image, includingmulti-GPU (graphics processing unit) rendering of geometry for anapplication by pretesting the geometry against screen regions, which maybe interleaved, in accordance with embodiments of the presentdisclosure.

FIG. 2 is a diagram of a multi-GPU architecture wherein multiple GPUscollaborate to render a single image, in accordance with one embodimentof the present disclosure.

FIG. 3 is a diagram of multiple graphics processing unit resourcesconfigured for multi-GPU rendering of geometry for an application bypretesting the geometry against screen regions, which may beinterleaved, in accordance with one embodiment of the presentdisclosure.

FIG. 4 is a diagram of a rendering architecture implementing a graphicspipeline that is configured for multi-GPU processing, such that multipleGPUs collaborate to render a single image, in accordance with oneembodiment of the present disclosure.

FIG. 5 is a flow diagram illustrating a method for graphics processingincluding multi-GPU rendering of geometry for an application bypretesting against interleaved screen regions before rendering, inaccordance with one embodiment of the present disclosure.

FIG. 6A is a diagram of a screen that is subdivided into quadrants whenperforming multi-GPU rendering, in accordance with one embodiment of thepresent disclosure.

FIG. 6B is a diagram of a screen that is subdivided into a plurality ofinterleaved regions when performing multi-GPU rendering, in accordancewith one embodiment of the present disclosure.

FIG. 7A is a diagram of a rendering command buffer that is shared bymultiple GPUs that collaborate to render a single image frame includinga pretesting of geometry portion and a rendering portion, in accordancewith one embodiment of the present disclosure.

FIG. 7B-1 illustrates an image including four objects rendered bymultiple GPUs, and shows the screen region responsibilities for each GPUwhen rendering the objects of the image, in accordance with oneembodiment of the present disclosure.

FIG. 7B-2 is a table illustrating the rendering performed by each GPUwhen rendering the four objects of FIG. 7B-1 , in accordance with oneembodiment of the present disclosure.

FIG. 7C is a diagram illustrating the performing of pretesting geometryand rendering of the geometry as performed by one or more GPUs whenrendering an image frame through collaboration of multiple GPUs (e.g.,the image of FIG. 7B-1 ), in accordance with one embodiment of thepresent disclosure.

FIG. 8A illustrates object testing against screen regions when multipleGPUs collaborate to render a single image, in accordance with oneembodiment of the present disclosure.

FIG. 8B illustrates testing of portions of an object against screenregions when multiple GPUs collaborate to render a single image, inaccordance with one embodiment of the present disclosure.

FIG. 9A-9C illustrates various strategies for assigning screen regionsto corresponding GPUs when multiple GPUs collaborate to render a singleimage, in accordance with one embodiment of the present disclosure.

FIG. 10 is a diagram illustrating various distributions of GPUassignment for performing geometry pretesting on a plurality of piecesof geometry, in accordance with embodiments of the present disclosure.

FIG. 11A is a diagram illustrating the pretesting and rendering ofgeometry of a previous image frame by a plurality of GPUs, and the useof statistics collected during rendering to influence the assignment ofpretesting of geometry of a current image frame to the plurality of GPUsin the current image frame, in accordance with one embodiment of thepresent disclosure.

FIG. 11B is a flow diagram illustrating a method for graphics processingincluding pretesting and rendering of geometry of a previous image frameby a plurality of GPUs, and the use of statistics collected duringrendering to influence the assignment of pretesting of geometry of acurrent image frame to the plurality of GPUs in the current image frame,in accordance with one embodiment of the present disclosure.

FIG. 12A is a diagram illustrating the use of shaders configured toperform both pretesting and rendering of geometry of an image frame intwo passes through a portion of the command buffer, in accordance withone embodiment of the present disclosure.

FIG. 12B is a flow diagram illustrating a method for graphics processingincluding performing both pretesting and rendering of geometry of animage frame using the same set of shaders in two passes through aportion of the command buffer, in accordance with one embodiment of thepresent disclosure.

FIG. 13A is a diagram illustrating the use of shaders configured toperform both geometry testing and rendering, wherein geometry test andrender performed for different sets of pieces of geometry areinterleaved using separate portions of a corresponding command buffer,in accordance with one embodiment of the present disclosure.

FIG. 13B is a flow diagram illustrating a method for graphics processingincluding interleaving pretesting and rendering of geometry of an imageframe for different sets of pieces geometry using separate portions of acorresponding command buffer, in accordance with one embodiment of thepresent disclosure.

FIG. 14 illustrates components of an example device that can be used toperform aspects of the various embodiments of the present disclosure.

DETAILED DESCRIPTION

Although the following detailed description contains many specificdetails for the purposes of illustration, anyone of ordinary skill inthe art will appreciate that many variations and alterations to thefollowing details are within the scope of the present disclosure.Accordingly, the aspects of the present disclosure described below areset forth without any loss of generality to, and without imposinglimitations upon, the claims that follow this description.

Generally speaking, there are limits to the performance that anindividual GPU can attain, e.g. deriving from the limits on how largethe GPU can be. To render even more complex scenes or use even morecomplex algorithms (e.g. materials, lighting, etc.) it is desirable touse multiple GPUs to render a single image, in embodiments of thepresent disclosure. In particular, various embodiments of the presentdisclosure describe methods and systems configured for performingmulti-GPU rendering of geometry for an application by pretesting thegeometry against screen regions, which may be interleaved. Multiple GPUscollaborate to generate an image. Responsibility for rendering isdivided between a plurality of the GPUs based on screen regions. Priorto rendering the geometry, the GPUs generate information regarding thegeometry and its relation to the screen regions. This allows the GPUs tomore efficiently render the geometry or avoid rendering it altogether.As an advantage, for example this allows the multiple GPUs to rendermore complex scenes and/or images in the same amount of time.

With the above general understanding of the various embodiments, exampledetails of the embodiments will now be described with reference to thevarious drawings.

Throughout the specification, the reference to “application” or “game”or “video game” or “gaming application” is meant to represent any typeof interactive application that is directed through execution of inputcommands. For illustration purposes only, an interactive applicationincludes applications for gaming, word processing, video processing,video game processing, etc. Further, the terms introduced above areinterchangeable.

Throughout the specification, various embodiments of the presentdisclosure are described for multi-GPU processing or rendering ofgeometry for an application using an exemplary architecture having fourGPUs. However, it is understood that any number of GPUs (e.g., two ormore GPUs) may collaborate when rendering geometry for an application.

FIG. 1 is a diagram of a system for performing multi-GPU processing whenrendering an image (e.g. image frame) for an application, in accordancewith one embodiment of the present disclosure. The system is configuredto provide gaming over a network between one or more cloud gamingservers, and more specifically is configured for the collaboration ofmultiple GPUs to render a single image of an application, in accordancewith embodiments of the present disclosure. Cloud gaming includes theexecution of a video game at the server to generate game rendered videoframes, which are then sent to a client for display. In particular,system 100 is configured for efficient multi-GPU rendering of geometryfor an application by pretesting against screen regions, which may beinterleaved, before rendering.

Although FIG. 1 illustrates the implementation of multi-GPU rendering ofgeometry between one or more cloud gaming servers of a cloud gamingsystem, other embodiments of the present disclosure provide forefficient multi-GPU rendering of geometry for an application byperforming region testing while rendering within a stand-alone system,such as a personal computer or gaming console that includes a high-endgraphics card having multiple GPUs.

It is also understood that the multi-GPU rendering of geometry may beperformed using physical GPUs, or virtual GPUs, or a combination ofboth, in various embodiments (e.g. in a cloud gaming environment orwithin a stand-alone system). For example, virtual machines (e.g.instances) may be created using a hypervisor of a host hardware (e.g.located at a data center) utilizing one or more components of a hardwarelayer, such as multiple CPUs, memory modules, GPUs, network interfaces,communication components, etc. These physical resources may be arrangedin racks, such as racks of CPUs, racks of GPUs, racks of memory, etc.,wherein the physical resources in the racks may be accessed using top ofrack switches facilitating a fabric for assembling and accessing ofcomponents used for an instance (e.g. when building the virtualizedcomponents of the instance). Generally, a hypervisor can presentmultiple guest operating systems of multiple instances that areconfigured with virtual resources. That is, each of the operatingsystems may be configured with a corresponding set of virtualizedresources supported by one or more hardware resources (e.g. located at acorresponding data center). For instance, each operating system may besupported with a virtual CPU, multiple virtual GPUs, virtual memory,virtualized communication components, etc. In addition, a configurationof an instance that may be transferred from one data center to anotherdata center to reduce latency. GPU utilization defined for the user orgame can be utilized when saving a user's gaming session. The GPUutilization can include any number of configurations described herein tooptimize the fast rendering of video frames for a gaming session. In oneembodiment, the GPU utilization defined for the game or the user can betransferred between data centers as a configurable setting. The abilityto transfer the GPU utilization setting enables for efficient migrationof game play from data center to data center in case the user connectsto play games from different geo locations.

System 100 provides gaming via a cloud game network 190, wherein thegame is being executed remote from client device 110 (e.g. thin client)of a corresponding user that is playing the game, in accordance with oneembodiment of the present disclosure. System 100 may provide gamingcontrol to one or more users playing one or more games through the cloudgame network 190 via network 150 in either single-player or multi-playermodes. In some embodiments, the cloud game network 190 may include aplurality of virtual machines (VMs) running on a hypervisor of a hostmachine, with one or more virtual machines configured to execute a gameprocessor module utilizing the hardware resources available to thehypervisor of the host. Network 150 may include one or morecommunication technologies. In some embodiments, network 150 may include5^(th) Generation (5G) network technology having advanced wirelesscommunication systems.

In some embodiments, communication may be facilitated using wirelesstechnologies. Such technologies may include, for example, 5G wirelesscommunication technologies. 5G is the fifth generation of cellularnetwork technology. 5G networks are digital cellular networks, in whichthe service area covered by providers is divided into small geographicalareas called cells. Analog signals representing sounds and images aredigitized in the telephone, converted by an analog to digital converterand transmitted as a stream of bits. All the 5G wireless devices in acell communicate by radio waves with a local antenna array and low powerautomated transceiver (transmitter and receiver) in the cell, overfrequency channels assigned by the transceiver from a pool offrequencies that are reused in other cells. The local antennas areconnected with the telephone network and the Internet by a highbandwidth optical fiber or wireless backhaul connection. As in othercell networks, a mobile device crossing from one cell to another isautomatically transferred to the new cell. It should be understood that5G networks are just an example type of communication network, andembodiments of the disclosure may utilize earlier generation wireless orwired communication, as well as later generation wired or wirelesstechnologies that come after 5G.

As shown, the cloud game network 190 includes a game server 160 thatprovides access to a plurality of video games. Game server 160 may beany type of server computing device available in the cloud, and may beconfigured as one or more virtual machines executing on one or morehosts. For example, game server 160 may manage a virtual machinesupporting a game processor that instantiates an instance of a game fora user. As such, a plurality of game processors of game server 160associated with a plurality of virtual machines is configured to executemultiple instances of one or more games associated with gameplays of aplurality of users. In that manner, back-end server support providesstreaming of media (e.g. video, audio, etc.) of gameplays of a pluralityof gaming applications to a plurality of corresponding users. That is,game server 160 is configured to stream data (e.g. rendered imagesand/or frames of a corresponding gameplay) back to a correspondingclient device 110 through network 150. In that manner, a computationallycomplex gaming application may be executing at the back-end server inresponse to controller inputs received and forwarded by client device110. Each server is able to render images and/or frames that are thenencoded (e.g. compressed) and streamed to the corresponding clientdevice for display.

For example, a plurality of users may access cloud game network 190 viacommunication network 150 using corresponding client devices 110configured for receiving streaming media. In one embodiment, clientdevice 110 may be configured as a thin client providing interfacing witha back end server (e.g. cloud game network 190) configured for providingcomputational functionality (e.g. including game title processing engine111). In another embodiment, client device 110 may be configured with agame title processing engine and game logic for at least some localprocessing of a video game, and may be further utilized for receivingstreaming content as generated by the video game executing at a back-endserver, or for other content provided by back-end server support. Forlocal processing, the game title processing engine includes basicprocessor based functions for executing a video game and servicesassociated with the video game. In that case, the game logic may bestored on the local client device 110 and is used for executing thevideo game.

Each of the client devices 110 may be requesting access to differentgames from the cloud game network. For example, cloud game network 190may be executing one or more game logics that are built upon a gametitle processing engine 111, as executed using the CPU resources 163 andGPU resources 365 of the game server 160. For instance, game logic 115 ain cooperation with game title processing engine 111 may be executing ongame server 160 for one client, game logic 115 b in cooperation withgame title processing engine 111 may be executing on game server 160 fora second client, . . . and game logic 115 n in cooperation with gametitle processing engine 111 may be executing on game server 160 for anNth client.

In particular, client device 110 of a corresponding user (not shown) isconfigured for requesting access to games over a communication network150, such as the internet, and for rendering for display images (e.g.image frame) generated by a video game executed by the game server 160,wherein encoded images are delivered to the client device 110 fordisplay in association with the corresponding user. For example, theuser may be interacting through client device 110 with an instance of avideo game executing on game processor of game server 160. Moreparticularly, an instance of the video game is executed by the gametitle processing engine 111. Corresponding game logic (e.g. executablecode) 115 implementing the video game is stored and accessible through adata store (not shown), and is used to execute the video game. Gametitle processing engine 111 is able to support a plurality of videogames using a plurality of game logics (e.g. gaming application), eachof which is selectable by the user.

For example, client device 110 is configured to interact with the gametitle processing engine 111 in association with the gameplay of acorresponding user, such as through input commands that are used todrive gameplay. In particular, client device 110 may receive input fromvarious types of input devices, such as game controllers, tabletcomputers, keyboards, gestures captured by video cameras, mice, touchpads, etc. Client device 110 can be any type of computing device havingat least a memory and a processor module that is capable of connectingto the game server 160 over network 150. The back-end game titleprocessing engine 111 is configured for generating rendered images,which is delivered over network 150 for display at a correspondingdisplay in association with client device 110. For example, throughcloud based services the game rendered images may be delivered by aninstance of a corresponding game (e.g. game logic) executing on gameexecuting engine 111 of game server 160. That is, client device 110 isconfigured for receiving encoded images (e.g. encoded from game renderedimages generated through execution of a video game), and for displayingthe images that are rendered on display 11. In one embodiment, display11 includes an HMD (e.g. displaying VR content). In some embodiments,the rendered images may be streamed to a smartphone or tablet,wirelessly or wired, direct from the cloud based services or via theclient device 110 (e.g. PlayStation® Remote Play).

In one embodiment, game server 160 and/or the game title processingengine 111 includes basic processor based functions for executing thegame and services associated with the gaming application. For example,game server 160 includes central processing unit (CPU) resources 163 andgraphics processing unit (GPU) resources 365 that are configured forperforming processor based functions include 2D or 3D rendering, physicssimulation, scripting, audio, animation, graphics processing, lighting,shading, rasterization, ray tracing, shadowing, culling, transformation,artificial intelligence, etc. In addition, the CPU and GPU group mayimplement services for the gaming application, including, in part,memory management, multi-thread management, quality of service (QoS),bandwidth testing, social networking, management of social friends,communication with social networks of friends, communication channels,texting, instant messaging, chat support, etc. In one embodiment, one ormore applications share a particular GPU resource. In one embodiment,multiple GPU devices may be combined to perform graphics processing fora single application that is executing on a corresponding CPU.

In one embodiment, cloud game network 190 is a distributed game serversystem and/or architecture. In particular, a distributed game engineexecuting game logic is configured as a corresponding instance of acorresponding game. In general, the distributed game engine takes eachof the functions of a game engine and distributes those functions forexecution by a multitude of processing entities. Individual functionscan be further distributed across one or more processing entities. Theprocessing entities may be configured in different configurations,including physical hardware, and/or as virtual components or virtualmachines, and/or as virtual containers, wherein a container is differentfrom a virtual machine as it virtualizes an instance of the gamingapplication running on a virtualized operating system. The processingentities may utilize and/or rely on servers and their underlyinghardware on one or more servers (compute nodes) of the cloud gamenetwork 190, wherein the servers may be located on one or more racks.The coordination, assignment, and management of the execution of thosefunctions to the various processing entities are performed by adistribution synchronization layer. In that manner, execution of thosefunctions is controlled by the distribution synchronization layer toenable generation of media (e.g. video frames, audio, etc.) for thegaming application in response to controller input by a player. Thedistribution synchronization layer is able to efficiently execute (e.g.through load balancing) those functions across the distributedprocessing entities, such that critical game engine components/functionsare distributed and reassembled for more efficient processing.

FIG. 2 is a diagram of an exemplary multi-GPU architecture 200 whereinmultiple GPUs collaborate to render a single image of a correspondingapplication, in accordance with one embodiment of the presentdisclosure. It is understood that many architectures are possible invarious embodiments of the present disclosure in which multiple GPUscollaborate to render a single image though not explicitly described orshown. For example, multi-GPU rendering of geometry for an applicationby performing region testing while rendering may be implemented betweenone or more cloud gaming servers of a cloud gaming system, or may beimplemented within a stand-alone system, such as a personal computer orgaming console that includes a high-end graphics card having multipleGPUs, etc.

The multi-GPU architecture 200 includes a CPU 163 and multiple GPUsconfigured for multi-GPU rendering of a single image for an application,and/or each image in a sequence of images for the application. Inparticular, CPU 163 and GPU resources 365 are configured for performingprocessor based functions include 2D or 3D rendering, physicssimulation, scripting, audio, animation, graphics processing, lighting,shading, rasterization, ray tracing, shadowing, culling, transformation,artificial intelligence, etc., as previously described.

For example, four GPUs are shown in GPU resources 365 of the multi-GPUarchitecture 200, though any number of GPUs may be utilized whenrendering images for an application. Each GPU is connected via a highspeed bus 220 to a corresponding dedicated memory, such as random accessmemory (RAM). In particular, GPU-A is connected to memory 210A (e.g.,RAM) via bus 220, GPU-B is connected to memory 210B (e.g., RAM) via bus220, GPU-C is connected to memory 210C (e.g., RAM) via bus 220, andGPU-D is connected to memory 210D (e.g., RAM) via bus 220.

Further, each GPU is connected to each other via bus 240 that dependingon the architecture may be approximately equal in speed or slower thanbus 220 used for communication between a corresponding GPU and itscorresponding memory. For example, GPU-A is connected to each of GPU-B,GPU-C, and GPU-D via bus 240. Also, GPU-B is connected to each of GPU-A,GPU-C, and GPU-D via bus 240. In addition, GPU-C is connected to each ofGPU-A, GPU-B, and GPU-D via bus 240. Further, GPU-D is connected to eachof GPU-A, GPU-B, and GPU-C via bus 240.

CPU 163 connects to each of the GPUs via a lower speed bus 230 (e.g.,bus 230 is slower than bus 220 used for communication between acorresponding GPU and its corresponding memory). In particular, CPU 163is connected to each of GPU-A, GPU-B, GPU-C, and GPU-D.

FIG. 3 is a diagram of graphics processing unit resources 365 configuredfor multi-GPU rendering of geometry for an image frame generated by anapplication by pretesting against screen regions, which may byinterleaved, before rendering, in accordance with one embodiment of thepresent disclosure. For example, game server 160 may be configured toinclude GPU resources 365 in the cloud game network 190 of FIG. 1 . Asshown, GPU resources 365 includes multiple GPUs, such as GPU 365 a, GPU365 b GPU 365 n. As previously described, various architectures mayinclude multiple GPUs collaborating to render a single image byperforming multi-GPU rendering of geometry for an application throughregion testing while rendering, such as implementing multi-GPU renderingof geometry between one or more cloud gaming servers of a cloud gamingsystem, or implementing multi-GPU rendering of geometry within astand-alone system, such as a personal computer or gaming console thatincludes a high-end graphics card having multiple GPUs, etc.

In particular, in one embodiment, game server 160 is configured toperform multi-GPU processing when rendering a single image of anapplication, such that multiple GPUs collaborate to render a singleimage, and/or render each of one or more images of a sequence of imageswhen executing an application. For example, game server 160 may includea CPU and GPU group that is configured to perform multi-GPU rendering ofeach of one or more images in a sequence of images of the application,wherein one CPU and GPU group could be implementing graphics and/orrendering pipelines for the application, in one embodiment. The CPU andGPU group could be configured as one or more processing devices. Aspreviously described, the GPU and GPU group may include CPU 163 and GPUresources 365, which are configured for performing processor basedfunctions include 2D or 3D rendering, physics simulation, scripting,audio, animation, graphics processing, lighting, shading, rasterization,ray tracing, shadowing, culling, transformation, artificialintelligence, etc.

GPU resources 365 are responsible and/or configured for rendering ofobjects (e.g. writing color or normal vector values for a pixel of theobject to multiple render targets—MRTs) and for execution of synchronouscompute kernels (e.g. full screen effects on the resulting MRTs); thesynchronous compute to perform, and the objects to render are specifiedby commands contained in rendering command buffers 325 that the GPU willexecute. In particular, GPU resources 365 is configured to renderobjects and perform synchronous compute (e.g. during the execution ofsynchronous compute kernels) when executing commands from the renderingcommand buffers 325, wherein commands and/or operations may be dependenton other operations such that they are performed in sequence.

For example, GPU resources 365 are configured to perform synchronouscompute and/or rendering of objects using one or more rendering commandbuffers 325 (e.g. rendering command buffer 325 a, rendering buffer 325 b. . . rendering command buffer 325 n). Each GPU in the GPU resources 365may have their own command buffers, in one embodiment. Alternatively,when substantially the same set of objects are being rendered by eachGPU (e.g., due to small size of the regions), the GPUs in GPU resources365 may use the same command buffer or the same set of command buffers.Further, each of the GPUs in GPU resources 365 may support the abilityfor a command to be executed by one GPU, but not by another. Forinstance, flags on a draw command or predication in the renderingcommand buffer allows a single GPU to execute one or more commands inthe corresponding command buffer, while the other GPUs will ignore thecommands. For example, rendering command buffer 325 a may support flags330 a, rendering command buffer 325 b may support flags 330 b . . .rendering command buffer 325 n may support flags 330 n.

Performance of synchronous compute (e.g. execution of synchronouscompute kernels) and rendering of objects are part of the overallrendering. For example, if the video game is running at 60 Hz (e.g. 60frames per second), then all object rendering and execution ofsynchronous compute kernels for an image frame typically must completewithin approximately 16.67 ms (e.g. one frame at 60 Hz). As previouslydescribed, operations performed when rendering objects and/or executingsynchronous compute kernels are ordered, such that operations may bedependent on other operations (e.g. commands in a rendering commandbuffer may need to complete execution before other commands in thatrendering command buffer can execute).

In particular, each of the rendering command buffers 325 containscommands of various types, including commands that affect acorresponding GPU configuration (e.g. commands that specify the locationand format of a render target), as well as commands to render objectsand/or execute synchronous compute kernels. For purposes ofillustration, synchronous compute performed when executing synchronizecompute kernels may include performing full screen effects when theobjects have all been rendered to one or more corresponding multiplerender targets (MRTs).

In addition, when GPU resources 365 render objects for an image frame,and/or execute synchronous compute kernels when generating the imageframe, the GPU resources 365 are configured via the registers of eachGPU 365 a, 365 b . . . 365 n. For example, GPU 365 a is configured viaits registers 340 (e.g. register 340 a, register 340 b . . . register340 n) to perform that rendering or compute kernel execution in acertain way. That is, the values stored in registers 340 define thehardware context (e.g. GPU configuration or GPU state) for GPU 365 awhen executing commands in rendering command buffers 325 used forrendering objects and/or executing synchronous compute kernels for animage frame. Each of the GPUs in GPU resources 365 may be similarlyconfigured, such that GPU 365 b is configured via its registers 350(e.g., register 350 a, register 350 b . . . register 350 n) to performthat rendering or compute kernel execution in a certain way; . . . andGPU 365 n is configured via its registers 370 (e.g., register 370 a,register 370 b . . . register 370 n) to perform that rendering orcompute kernel execution in a certain way.

Some examples of GPU configuration include the location and format ofrender targets (e.g. MRTs). Also, other examples of GPU configurationinclude operating procedures. For instance, when rendering an object,the Z-value of each pixel of the object can be compared to the Z-bufferin various ways. For example, the object pixel is written only if theobject Z-value matches the value in the Z-buffer. Alternatively, theobject pixel could be written only if the object Z-value is the same orless than the value in the Z-buffer. The type of test being performed isdefined within the GPU configuration.

FIG. 4 is a simplified diagram of a rendering architecture implementinga graphics pipeline 400 that is configured for multi-GPU processing,such that multiple GPUs collaborate to render a single image, inaccordance with one embodiment of the present disclosure. The graphicspipeline 400 is illustrative of the general process for rendering imagesusing 3D (three dimensional) polygon rendering processes. The graphicspipeline 400 for a rendered image outputs corresponding colorinformation for each of the pixels in a display, wherein the colorinformation may represent texture and shading (e.g., color, shadowing,etc.). Graphics pipeline 400 may be implementable within the clientdevice 110, game server 160, game title processing engine 111, and/orGPU resources 365 of FIGS. 1 and 3 . That is, various architectures mayinclude multiple GPUs collaborating to render a single image byperforming multi-GPU rendering of geometry for an application throughregion testing while rendering, such as implementing multi-GPU renderingof geometry between one or more cloud gaming servers of a cloud gamingsystem, or implementing multi-GPU rendering of geometry within astand-alone system, such as a personal computer or gaming console thatincludes a high-end graphics card having multiple GPUs, etc.

As shown, the graphics pipeline receives input geometries 405. Forexample, the geometry processing stage 410 receives the input geometries405. For example, the input geometries 405 may include vertices within a3D gaming world, and information corresponding to each of the vertices.A given object within the gaming world can be represented using polygons(e.g., triangles) defined by vertices, wherein the surface of acorresponding polygon is then processed through the graphics pipeline400 to achieve a final effect (e.g., color, texture, etc.). Vertexattributes may include normal (e.g., which direction is perpendicular tothe geometry at that location), color (e.g., RGB—red, green, and bluetriple, etc.), and texture coordinate/mapping information.

The geometry processing stage 410 is responsible for (and capable of)both vertex processing (e.g. via a vertex shader) and primitiveprocessing. In particular, the geometry processing stage 410 may outputsets of vertices that define primitives and deliver them to the nextstage of the graphics pipeline 400, as well as positions (to be precise,homogeneous coordinates) and various other parameters for thosevertices. The positions are placed in the position cache 450 for accessby later shader stages. The other parameters are placed in the parametercache 460, again for access by later shader stages.

Various operations may be performed by the geometry processing stage410, such as performing lighting and shadowing calculations for theprimitives and/or polygons. In one embodiment, as the geometry stage iscapable of processing of primitives, it can perform backface culling,and/or clipping (e.g. testing against the view frustum), therebyreducing the load on downstream stages (e.g., rasterization stage 420,etc.). In another embodiment, the geometry stage may generate primitives(e.g. with functionality equivalent to a traditional geometry shader).

The primitives output by the geometry processing stage 410 are fed intothe rasterization stage 420 that converts the primitives into a rasterimage composed of pixels. In particular, the rasterization stage 420 isconfigured to project objects in the scene to a two-dimensional (2D)image plane defined by the viewing location in the 3D gaming world(e.g., camera location, user eye location, etc.). At a simplistic level,the rasterization stage 420 looks at each primitive and determines whichpixels are affected by the corresponding primitive. In particular, therasterizer 420 partitions the primitives into pixel sized fragments,wherein each fragment corresponds to a pixel in the display. It isimportant to note that one or more fragments may contribute to the colorof a corresponding pixel when displaying an image.

As previously described, additional operations may also be performed bythe rasterization stage 420 such as clipping (identify and disregardfragments that are outside the viewing frustum) and culling (disregardfragments that are occluded by closer objects) to the viewing location.With reference to clipping, the geometry processing stage 410 and/orrasterization stage 420 may be configured to identify and disregardprimitives that are outside the viewing frustum as defined by theviewing location in the gaming world.

The pixel processing stage 430 uses the parameters created by thegeometry processing stage, as well as other data, to generate valuessuch as the resulting color of the pixel. In particular, the pixelprocessing stage 430 at its core performs shading operations on thefragments to determine how the color and brightness of a primitivevaries with available lighting. For example, pixel processing stage 430may determine depth, color, normal and texture coordinates (e.g.,texture details) for each fragment, and may further determineappropriate levels of light, darkness, and color for the fragments. Inparticular, pixel processing stage 430 calculates the traits of eachfragment, including color and other attributes (e.g., z-depth fordistance from the viewing location, and alpha values for transparency).In addition, the pixel processing stage 430 applies lighting effects tothe fragments based on the available lighting affecting thecorresponding fragments. Further, the pixel processing stage 430 mayapply shadowing effects for each fragment.

The output of the pixel processing stage 430 includes processedfragments (e.g., texture and shading information) and is delivered tothe output merger stage 440 in the next stage of the graphics pipeline400. The output merger stage 440 generates a final color for the pixel,using the output of the pixel processing stage 430, as well as otherdata, such as a value already in memory. For example, the output mergerstage 440 may perform optional blending of values between fragmentsand/or pixels determined from the pixel processing stage 430, and valuesalready written to an MRT for that pixel.

Color values for each pixel in the display may be stored in a framebuffer (not shown). These values are scanned to the corresponding pixelswhen displaying a corresponding image of the scene. In particular, thedisplay reads color values from the frame buffer for each pixel,row-by-row, from left-to-right or right-to-left, top-to-bottom orbottom-to-top, or any other pattern, and illuminates pixels using thosepixel values when displaying the image.

With the detailed description of the cloud game network 190 (e.g. in thegame server 160) and the GPU resources 365 of FIGS. 1-3 , flow diagram500 of FIG. 5 illustrates a method for graphics processing whenimplementing multi-GPU rendering of geometry for an image framegenerated by an application by pretesting the geometry againstinterleaved screen regions before rendering, in accordance with oneembodiment of the present disclosure. In that manner, multiple GPUresources are used to efficiently perform rendering of objects whenexecuting an application. As previously described, various architecturesmay include multiple GPUs collaborating to render a single image byperforming multi-GPU rendering of geometry for an application throughregion testing while rendering, such as within one or more cloud gamingservers of a cloud gaming system, or within a stand-alone system, suchas a personal computer or gaming console that includes a high-endgraphics card having multiple GPUs, etc.

At 510, the method includes rendering graphics for an application usinga plurality of graphics processing units (GPUs) that collaborate togenerate an image. In particular, multi-GPU processing is performed whenrendering a single image frame and/or each of one or more image framesof a sequence of image frames for a real-time application.

At 520, the method includes dividing responsibility for the renderinggeometry of the graphics between the plurality of GPUs based on aplurality of screen regions. That is, each GPU has a correspondingdivision of the responsibility (e.g., corresponding screen region) whichis known to all the GPUs. More specifically, each of the GPUs isresponsible for rendering geometry in a corresponding set of screenregions of the plurality of screen regions, wherein the correspondingset of screen regions includes one or more screen regions. For example,a first GPU has the first division of responsibility for renderingobjects in a first set of screen regions. Also, a second GPU has asecond division of responsibility for rendering objects in a second setof screen regions. This is repeatable for remaining GPUs.

At 530, the method includes assigning a first GPU a first piece ofgeometry of an image frame generated during execution an application forgeometry testing. For example, an image frame may include one or moreobjects, wherein each object may be defined by one or more pieces ofgeometry. That is, geometry pretesting and rendering are performed on apiece of geometry that is an entire object, in one embodiment. In otherembodiments, geometry pretesting and rendering are performed on a pieceof geometry that is a portion of an entire object.

For example, each of the plurality of GPUs is assigned to acorresponding portion of the geometry associated with an image frame. Inparticular, every portion of the geometry is assigned to a correspondingGPU for purposes of geometry pretesting. The geometry may be evenlyassigned between the plurality of GPUs, in one embodiment. For example,if there are four GPUs in the plurality, then each of the GPUs mayprocess a quarter of the geometry in an image frame. In otherembodiments, the geometry may be unevenly assigned between the pluralityof GPUs. For example, in the example of using four GPUs for multi-GPUrendering of an image frame, one GPU may process more geometry of animage frame than another GPU.

At 540, the method includes performing geometry pretesting at the firstGPU to generate information for how the piece of geometry relates to theplurality of screen regions. In particular, the first GPU generatesinformation for the piece of geometry, and how it relates to each of theplurality of screen regions. For example, geometry pretesting by thefirst GPU may determine whether or not the piece of geometry overlaps aparticular screen region that is assigned to a corresponding GPU forobject rendering. The first piece of geometry may overlap screen regionsfor which other GPUs are responsible for object rendering, and/or mayoverlap screen regions for which the first GPU is responsible for objectrendering. In one embodiment, the geometry testing is performed byshaders in a corresponding command buffer executed by the first GPUbefore performing rendering of the geometry by any of the plurality ofGPUs. In other embodiments, the geometry testing is performed byhardware, e.g. in the rasterization stage 420 of the graphics pipeline400.

Geometry pretesting is typically in embodiments performed simultaneouslyfor all geometry of a corresponding image frame by the plurality ofGPUs. That is, each GPU performs geometry pretesting for its portion ofthe geometry of a corresponding image frame. In that manner, geometrypretesting by the GPUs allows each GPU to know which pieces of geometryto render, and also which pieces of geometry to skip. In particular,when a corresponding GPU performs geometry pretesting, it tests itsportion of the geometry against the screen regions of each of theplurality of GPUs used for rendering the image frame. For example, ifthere are four GPUs, then each GPU may perform geometry testing on aquarter of the geometry of the image frame, especially if the geometryis assigned evenly to the GPUs for purposes of geometry testing. Assuch, even though each GPU is only performing geometry pretesting forits portion of the geometry of a corresponding image frame, becausegeometry pretesting is typically in embodiments performed simultaneouslyfor all geometry of the image frame across the plurality of GPUs, theinformation generated indicates how all the geometry (e.g. pieces ofgeometry) in the image frame relates to screen regions of all GPUs,wherein screen regions are each assigned to a corresponding GPU forobject rendering, and/or wherein rendering may be performed on pieces ofgeometry (e.g. an entire object or a portion of an object).

At 550, the method includes using the information at each of theplurality of GPUs when rendering the piece of geometry (e.g. to includefully rendering the piece of geometry or skipping the rendering of thatpiece of geometry). That is, the information is used at each of theplurality of GPUs to render the piece of geometry, wherein test results(e.g. information) of the geometry are sent to other GPUs, such that theinformation is known to each of the GPUs. For example, the geometry(e.g. pieces of geometry) in the image frame is typically in embodimentsrendered simultaneously by the plurality of GPUs. In particular, when apiece of geometry overlaps any screen region assigned to a correspondingGPU for object rendering, that GPU will render that piece of geometrybased on the information. On the other hand, when the piece of geometrydoes not overlap any screen region assigned to the corresponding GPU forobject rendering, that GPU can skip rendering of that piece of geometrybased on the information. As such, the information allows all GPUs tomore efficiently render geometry in an image frame, and/or to avoidrendering that geometry altogether. For example, the rendering may beperformed by shaders in a corresponding command buffer as executed bythe plurality of GPUs. As will be described more fully below in FIGS.7A, 12A, and 13A, the shaders may be configured to perform one or bothof geometry testing and/or rendering, based on corresponding GPUconfigurations.

In some architectures, if a corresponding rendering GPU receivescorresponding information in time to use it, that GPU will use theinformation when deciding which geometry to render within acorresponding image, in accordance with one embodiment of the presentdisclosure. That is, the information may be taken as a hint. Otherwise,the rendering GPU will process the piece of geometry as it ordinarilywould. Using the example wherein the information may indicate whetherthe geometry overlaps any screen region assigned to a rendering GPU(e.g. a second GPU), if the information indicates there is nooverlapping of the geometry, the rendering GPU may skip rendering thegeometry entirely. Also, if only pieces of the geometry do not overlap,the second GPU may skip rendering of at least those pieces of geometrythat do not overlap any of the screen regions assigned to the second GPUfor object rendering. On the other hand, the information may indicatethat there is overlapping for the geometry, in which case the second orrendering GPU would render the geometry. Also, the information mayindicate that certain pieces of the geometry overlap any of the screenregions assigned to the second or rendering GPU for object rendering. Inthat case, the second or rendering GPU would render only those pieces ofthe geometry that overlap. In still another embodiment, if there is noinformation, or if the information is not generated or received in time,the second GPU would perform rendering normally (e.g., render thegeometry). As such, information provided as a hint may increase overallefficiency of the graphics processing system if received in time. If theinformation is not received in time, the graphics processing system willstill operate properly in the absence of such information.

In one embodiment, one GPU (e.g. a pretest GPU) is dedicated toperforming geometry pretesting to generate information. That is, thededicated GPU is not used for rendering objects (e.g. pieces ofgeometry) in the corresponding image frame. Specifically, graphics foran application are rendered using a plurality of GPUs, as previouslydescribed. Responsibility for rendering geometry of the graphics isdivided between the plurality of GPUs based on a plurality of screenregions, which may be interleaved, wherein each GPU has a correspondingdivision of the responsibility which is known to the plurality of GPUs.Geometry testing is performed at a pretest GPU on a plurality of piecesof geometry of an image frame generated by an application in order togenerate information regarding each piece of geometry and its relationto each of the plurality of screen regions. The plurality of pieces ofgeometry are rendered at each of the plurality of GPUs using theinformation generated for each of the plurality of pieces of geometry.That is, the information is used when rendering each of the pieces ofgeometry by a corresponding rendering GPU from the GPUs used to renderthe image frame.

FIGS. 6A-6B show renderings to screens that are subdivided into regionsand sub-regions, purely for purposes of illustration. It is understoodthat the number of subdivided regions and/or sub-regions is selectablefor efficient multi-GPU processing of an image and/or each of one ormore images of a sequence of images. That is, the screen may besubdivided into two or more regions, wherein each region may be furtherdivided into sub-regions. In one embodiment of the present disclosure,the screen is subdivided into four quadrants as shown in FIG. 6A. Inanother embodiment of the present disclosure, the screen is subdividedinto a larger number of interleaved regions as shown in FIG. 6B. Thediscussion of FIGS. 6A-6B below is intended to illustrate theinefficiencies that arise when performing multi-GPU rendering to aplurality of screen regions to which a plurality of GPUs are assigned;FIGS. 7A-7C and FIGS. 8A-8B show more efficient rendering, according tosome embodiments of the invention.

In particular, FIG. 6A is a diagram of a screen 610A that is subdividedinto quadrants (e.g. four regions) when performing multi-GPU rendering.As shown, screen 610A is subdivided into four quadrants (e.g. A, B, C,and D). Each quadrant is assigned to one of the four GPUs [GPU-A, GPU-B,GPU-C, and GPU-D], in a one-to-one relationship. For example, GPU-A isassigned to quadrant A, GPU-B is assigned to quadrant B, GPU-C isassigned to quadrant C, and GPU-D is assigned to quadrant D.

The geometry can be culled. For example, CPU 163 can check a boundingbox against each quadrant's frustum, and request each GPU to render onlythe objects that overlap its corresponding frustum. The result is thateach GPU is responsible for rendering only a portion of the geometry.For purposes of illustration, screen 610 shows pieces of geometry,wherein each piece is a corresponding object, wherein screen 610 showsobjects 611-617 (e.g. pieces of geometry). GPU-A will render no objects,as no objects overlap Quadrant A. GPU-B will render objects 615 and 616(as a portion of object 615 is present in Quadrant B, the CPU's cullingtest will correctly conclude that GPU-B must render it). GPU-C willrender objects 611 and 612. GPU-D will render objects 612, 613, 614, 615and 617.

In FIG. 6A, when the screen 610A is divided into quadrants A-D, theamount of work that each GPU must perform may be very different, as adisproportionate amount of geometry may be in one quadrant in somesituations. For example, quadrant A does not have any pieces ofgeometry, whereas quadrant D has five pieces of geometry, or at leastportions of at least five pieces of geometry. As such, GPU-A assigned toquadrant A would be idle, while GPU-D assigned to quadrant D would bedisproportionately busy when rendering objects in the correspondingimage.

FIG. 6B illustrates another technique when subdividing a screen intoregions. In particular, rather than subdividing into quadrants, screen610B is subdivided into a plurality of interleaved regions whenperforming multi-GPU rendering of a single image or each of one or moreimages in a sequence of images. In that case, screen 610B is subdividedinto a larger number of interleaved regions (e.g. greater than the fourquadrants), while using the same amount of GPUs for rendering (e.g.four). The objects (611-617) shown in screen 610A are also shown inscreen 610B in the same corresponding locations.

In particular, four GPUs (e.g. GPU-A, GPU-B, GPU-C, and GPU-D) are usedto render an image for a corresponding application. Each of the GPUs isresponsible for rendering geometry overlapping a corresponding region.That is, each GPU is assigned to a corresponding set of regions. Forexample, GPU-A is responsible for each of the regions labeled A in acorresponding set, GPU-B is responsible for each of regions labeled B ina corresponding set, GPU-C is responsible for each of regions labeled Cin a corresponding set, and GPU-D is responsible for each of regionslabeled D in a corresponding set.

Further, the regions are interleaved in a particular pattern. Because ofthe interleaving (and higher number) of regions, the amount of work thateach GPU must perform may be much more balanced. For example, thepattern of interleaving of screen 610B includes alternating rowsincluding regions A-B-A-B and so on, and regions C-D-C-D and so on.Other patterns of interleaving the regions is supported in embodimentsof the present disclosure. For example, patterns may include repeatedsequences of regions, evenly distributed regions, uneven distribution ofregions, repeatable rows of sequences of regions, random sequences ofregions, random rows of sequences of regions, etc.

Choosing the number of regions is important. For example, if thedistribution of regions is too fine (e.g. the number of regions is toogreat to be optimal), each GPU must still process most or all of thegeometry. For example, it may be difficult to check object boundingboxes against all of the regions that a GPU is responsible for. Also,even if bounding boxes can be checked in a timely manner, due to smallregions size, the result will be that each GPU likely has to processmost of the geometry because every object in an image overlaps at leastone of the regions of each of the GPUs (e.g. a GPU processes an entireobject even though only a portion of the object overlaps at least oneregion in a set of regions assigned to that GPU).

As a result, choosing the number of regions, the pattern ofinterleaving, etc. is important. Choosing too few or too many regions,or too few regions or too many regions for interleaving, or choosing aninefficient pattern for interleaving may lead to inefficiencies whenperforming GPU processing (e.g. each GPU processing most or all of thegeometry). In those cases, even though there are multiple GPUs forrendering an image, due to GPU inefficiencies, there is not the abilityto support a corresponding increase in both screen pixel count anddensity of geometry (i.e. four GPUs can't write four times the pixelsand process four times the vertices or primitives). The followingembodiments target improvements in culling strategy (FIGS. 7A-7C) andgranularity of culling (FIGS. 8A-8B), among other advances.

FIGS. 7A-7C are diagrams illustrating the use of multiple GPUs to rendera single image, and/or each of at least one or more images in a sequenceof images, in embodiments of the present disclosure. The selection offour GPUs is made purely for ease of illustrating multi-GPU renderingwhen rendering an image while executing an application, and it isunderstood that any number of GPUs may be used for multi-GPU renderingin various embodiments.

In particular, FIG. 7A is a diagram of a rendering command buffer 700Athat is shared by multiple GPUs that collaborate to render a singleimage frame, in accordance with one embodiment of the presentdisclosure. That is, in the present example the multiple GPUs each usethe same rendering command buffer (e.g., buffer 700A), and each of theGPUs execute all commands in the rendering command buffer. A pluralityof commands (complete set) is loaded into rendering command buffer 700A,and is used for rendering a corresponding image frame. It is understoodthat one or more rendering command buffers may be used to generate acorresponding image frame. In one example, the CPU generates one or moredraw calls for the image frame, wherein the draw calls include commandsplaced into one or more rendering command buffers for execution by oneor more GPUs of the GPU resources 365 of FIG. 3 when performingmulti-GPU rendering of a corresponding image. In some implementations,the CPU 163 may request one or more GPUs to generate all or some of thedraw calls used for rendering a corresponding image. Further, the entireset of commands may be shown in FIG. 7A that are contained within therendering command buffer 700A, or FIG. 7A may show a portion of theentire set of commands contained within the rendering command buffer700A.

GPUs typically in embodiments render simultaneously when performingmulti-GPU rendering of an image or each of one or more images in asequence of images. Rendering of an image can be broken down intomultiple phases. In each of the phases, the GPUs need to besynchronized, such that a faster GPU must wait until the slower GPUscomplete. The commands shown in FIG. 7A for the rendering command buffer700A shows one phase. Though commands for only one phase is shown inFIG. 7A, the rendering command buffer 700A may include commands for oneor more phases when rendering an image, FIG. 7A only shows a portion ofall the commands, such that commands for the other phases are not shown.In the piece of the rendering command buffer 700A shown in FIG. 7A thatillustrates one phase, there are four objects to be rendered (e.g.,object 0, object 1, object 2, and object 3), as is shown in FIG. 7B-1 .

As shown, the piece of the rendering command buffer 700A shown in FIG.7A includes commands for geometry testing, rendering of objects (e.g.pieces of geometry) and commands for configuring state of the one ormore rendering GPUs that are executing commands from rendering commandbuffer 700A. For purposes of illustration only, the piece of renderingcommand buffer 700A shown in FIG. 7A includes commands (710-728) usedfor geometry pretesting, rendering objects and/or executing synchronouscompute kernels when rendering a corresponding image for a correspondingapplication. In some implementations, the geometry pretesting, andrendering of objects for that image and/or the execution of synchronouscompute kernels must be performed within a frame period. Two processingsections are shown in the rendering command buffer 700A. In particular,processing section 1 includes pretesting or geometry testing 701, andsection 2 includes rendering 702.

Section 1 includes performing geometry testing 701 of objects in theimage frame, wherein each object may be defined by one or more pieces ofgeometry. Pretesting or geometry testing 701 may be performed by one ormore shaders. For example, each GPU used in multi-GPU rendering of acorresponding image frame is assigned a portion of the geometry of theimage frame to perform geometry testing, wherein every portion may beassigned for pretesting, in one embodiment. The assigned portion mayinclude one or more pieces of geometry, wherein a piece may include anentire object, or may include a portion of an object (e.g., vertex,primitive, etc.). In particular, geometry testing is performed on apiece of geometry to generate information on how that piece of geometryrelates to each of the plurality of screen regions. For example,geometry testing may determine whether a piece of geometry overlaps aparticular screen region assigned to a corresponding GPU for objectrendering.

As shown in FIG. 7A, geometry testing 701 (e.g., pretesting of geometry)of section 1 includes commands for configuring a state of the one ormore GPUs executing commands from the rendering command buffer 700A, andcommands for performing geometry testing. In particular, the GPU stateof each GPU is configured before the GPUs perform geometry testing oncorresponding objects. For example, commands 710, 713, and 715 are eachused for configuring a GPU state of the one or more GPUs for purposes ofexecuting commands for geometry testing. As shown, command 710configures GPU state so that geometry testing commands 711-712 can beproperly performed, wherein command 711 performs geometry testing onobject 0, and command 712 performs geometry testing on object 1.Similarly, command 713 configures GPU state so that geometry testingcommand 714 can perform geometry testing for object 2. Also, command 715configures GPU state so that geometry testing command 716 can performgeometry testing for object 3. It is understood that a GPU state may beconfigured for one or more geometry testing commands (e.g., testingcommands 711 and 712).

As previously described, values stored in the registers define thehardware context (e.g. GPU configuration) for the corresponding GPU whenexecuting commands in the rendering command buffer 700A used forgeometry testing and/or rendering objects and/or executing synchronouscompute kernels for a corresponding image. As shown, the GPU state maybe modified throughout the processing of commands in the renderingcommand buffer 700A, each subsequent section of commands may be used forconfiguring the GPU state. As applied to FIG. 7A, as well as throughoutthe specification when referring to setting GPU state, the GPU state maybe set in a variety of ways. For example, the CPU or GPU could set avalue in random access memory (RAM), wherein the GPU would check thevalue in RAM. In another example, the state could be internal to theGPU, such as when a command buffer is called as a subroutine twice withinternal GPU state being different between the two subroutine calls.

Section 2 includes performing rendering 702 of objects in the imageframe, wherein pieces of geometry are rendered). Rendering 702 may beperformed by one or more shaders in the command buffer 700A. As shown inFIG. 7A, rendering 702 of section 2 includes commands for configuring astate of the one or more GPUs executing commands from the renderingcommand buffer 700A, and commands for performing the rendering. Inparticular, the GPU state of each GPU is configured before the GPUsrender corresponding objects (e.g. pieces of geometry). For example,commands 721, 723, 725, and 727 are each used for configuring a GPUstate of the one or more GPUs for purposes of executing commands forrendering. As shown, command 721 configures GPU state so that renderingcommand 722 can render object 0; command 723 configures GPU state sothat rendering command 724 can render object 1; command 725 configuresGPU state so that rendering command 726 can render object 2; and command727 configures GPU state so that rendering command 728 can render object3. Though FIG. 7A shows that GPU state is configured for each renderingcommand (e.g., render object 0, etc.), it is understood that a GPU statemay be configured for one or more rendering commands.

As previously described, each GPU used in multi-GPU rendering of acorresponding image frame renders corresponding pieces of geometry basedon the information generated during geometry pretesting. Specifically,the information known to each of the GPUs provides relationships betweenobjects and screen regions. When rendering corresponding pieces ofgeometry, a GPU may use that information if received in a timely fashionfor purposes of efficiently rendering those pieces of geometry.Specifically, as indicated by the information, when a piece of geometryoverlaps any screen region or regions assigned to a corresponding GPUfor object rendering, that GPU performs rendering for that piece ofgeometry. On the other hand, the information may indicate that a firstGPU should skip rendering a piece of geometry entirely (e.g., the pieceof geometry does not overlap any screen region that the first GPU isassigned responsibility for object rendering). In that manner, each GPUonly renders pieces of geometry that overlap the screen region orregions to which it is responsible for object rendering. As such, theinformation is provided as a hint to each of the GPUs, such that theinformation is considered by each GPU that is performing renderingpieces of geometry if received before rendering begins. In oneembodiment, rendering proceeds normally if the information is notreceived in time, such as the corresponding piece of geometry isrendered fully by a corresponding GPU regardless of whether that pieceof geometry overlaps any screen regions that are assigned to the GPU forobject rendering.

For purposes of illustration only, four GPUs are dividing up acorresponding screen into regions between them. As previously described,each GPU is responsible for rendering objects in a corresponding set ofregions, wherein the corresponding set includes one or more regions. Inone embodiment, rendering command buffer 700A is shared by multiple GPUsthat collaborate to render a single image. That is, the GPUs used formulti-GPU rendering of a single image or each of one or more images in asequence of images share a common command buffer. In another embodiment,each GPU might have its own command buffer.

Alternatively, in still another embodiment each of the GPUs might berendering somewhat different sets of objects. This may be the case whenit can be determined that a specific GPU does not need to render aspecific object because it does not overlap its corresponding screenregions, such as in a corresponding set. The multiple GPUs can still usethe same command buffer (e.g., sharing one command buffer), as long asthe command buffer supports the ability for a command to be executed byone GPU but not by another, as previously described. For example,execution of a command in the shared rendering command buffer 700A maybe limited to one of the rendering GPUs. This could be accomplished in avariety of ways. In another example, flags may be used on acorresponding command to indicate which GPUs should execute it. Also,predication may be implemented in the rendering command buffer usingbits to say which GPU does what under which condition. An example ofpredication includes—“If this is GPU-A, then skip the following Xcommands”.

In still another embodiment, as substantially the same set of objects isbeing rendered by each GPU, the multiple GPUs may still use the samecommand buffer. For example, when the regions are relatively small, eachGPU may still render all of the objects, as previously described.

FIG. 7B-1 illustrates a screen 700B showing an image including fourobjects that are rendered by multiple GPUs using the rendering commandbuffer 700A of FIG. 7A, in accordance with one embodiment of the presentdisclosure. Multi-GPU rendering of geometry is performed for anapplication by pretesting the geometry against screen regions, which maybe interleaved, before rendering pieces of geometry corresponding toobjects in an image frame, in accordance with one embodiment of thepresent disclosure.

In particular, responsibility for rendering of geometry is divided up byscreen region between the multiple GPUs, wherein the plurality of screenregions is configured to reduce imbalance of rendering time between theplurality of GPUs. For example, screen 700B shows the screen regionresponsibilities for each GPU when rendering the objects of the image.Four GPUs (GPU-A, GPU-B, GPU-C, and GPU-D) are used for renderingobjects in the image shown in screen 700B. Screen 700B is divided morefinely than by quadrants as shown in FIG. 6A, in an effort to balancepixel and vertex load between the GPUs. In addition, screen 700B isdivided into regions, that may be interleaved. For example, theinterleaving includes multiple rows of regions. Each of rows 731 and 733includes region A alternating with region B. Each of rows 732 and 734includes region C alternating with region D. More particularly, rowsincluding regions A and B alternate with rows including regions C and D,in a pattern.

As previously described, to achieve GPU processing efficiency varioustechniques may be used when dividing the screen into regions, such asincreasing or decreasing the number of regions (e.g., to choose thecorrect amount of regions), interleaving regions, increasing ordecreasing the number of regions for interleaving, selecting aparticular pattern when interleaving regions and/or sub-regions, etc. Inone embodiment, each of the plurality of screen regions is uniformlysized. In one embodiment, each of the plurality of screen regions is notuniform in size. In still another embodiment, the number and sizing of aplurality of screen regions changes dynamically.

Each of the GPUs is responsible for rendering of objects in acorresponding set of regions, wherein each set may include one or moreregions. As such, GPU-A is responsible for rendering of objects in eachof the A regions in a corresponding set, GPU-B is responsible forrendering of objects in each of the B regions in a corresponding set,GPU-C is responsible for rendering of objects in each of the C regionsin a corresponding set, and GPU-D is responsible for rendering ofobjects in each of the D regions in a corresponding set. There mightalso be GPUs that have other responsibilities, such that they may notperform rendering (e.g., perform asynchronous compute kernels thatexecute over multiple frame periods, perform culling for the renderingGPUs, etc.).

The amount of rendering to be performed is different for each GPU. FIG.7B-2 illustrates a table showing the rendering performed by each GPUwhen rendering the four objects of FIG. 7B-1 , in accordance with oneembodiment of the present disclosure. As shown in the table, aftergeometry pretesting, it may be determined that object 0 is rendered byGPU-B; that object 1 is rendered by GPU-C and GPU-D; that object 2 isrendered by GPU-A, GPU-B, and GPU-D; and that object 3 is rendered byGPU-B, GPU-C, and GPU-D. There may still be some unbalanced rendering,as GPU A needs to render object 2 only, and GPU D needs to renderobjects 1, 2 and 3. However, overall, with interleaving of screenregions, the rendering of objects within an image is reasonably balancedacross the multiple GPUs used for multi-GPU rendering of an image, orrendering of each of one or more images in a sequence of images.

FIG. 7C is a diagram illustrating the rendering of each object asperformed by each GPU when multiple GPUs collaborate to render a singleimage frame, such as the image frame 700B shown in FIG. 7B-1 , inaccordance with one embodiment of the present disclosure. In particular,FIG. 7C shows the rendering process of objects 0-3 as performed by eachof the four GPUs (e.g., GPU-A, GPU-B, GPU-C, and GPU-D) using the sharedrendering command buffer 700A of FIG. 7A.

In particular, two rendering timing diagrams are shown with respect to atimeline 740. Rendering timing diagram 700C-1 shows multi-GPU renderingof objects 0-3 of a corresponding image in one phase of rendering,wherein each of the GPUs perform rendering in the absence of anyinformation regarding the overlap between objects 0-3 and the screenregions. Rendering timing diagram 700C-2 shows multi-GPU rendering ofobjects 0-3 of the corresponding image in the same phase of rendering,wherein information generated during geometry testing of screen regions(e.g. performed before rendering) are shared with each of the GPUs usedfor rendering objects 0-3 through a corresponding GPU pipeline. Each ofrendering timing diagrams 700C-1 and 700C-2 show the time taken by eachGPU to process each piece of geometry (e.g., perform geometry testingand rendering). In one embodiment, a piece of geometry is an entireobject. In another embodiment, a piece of geometry may be a portion ofan object. For purposes of illustration, the example of FIG. 7C showsthe rendering of pieces of geometry, wherein each piece of geometrycorresponds to an object (e.g. in its entirety). In each of therendering timing diagrams 700C-1 and 700C-2 objects (e.g. pieces ofgeometry) that have no geometry (e.g. a primitive of the object) thatoverlaps at least one screen region (e.g. in a corresponding set ofregions) of a corresponding GPU are represented by boxes drawn withdashed lines. On the other hand, objects that have geometry thatoverlaps at least one screen region (e.g. in a corresponding set ofregions) of a corresponding GPU are represented by boxes drawn withsolid lines.

Rendering timing diagram 700C-1 shows rendering of objects 0-3 using thefour GPUs (e.g. GPU-A, GPU-B, GPU-C, and GPU-D). Vertical line 755 aindicates the start of the phase of rendering for the objects, andvertical line 755 b shows the end of the phase of rendering for theobjects in rendering timing diagram 700C-1. The start and end pointsalong timeline 740 for the phase of rendering shown representsynchronization points, wherein each of the four GPUs are synchronizedwhen executing a corresponding GPU pipeline. For instance, at verticalline 755 b indicating the end of the phase of rendering, all GPUs mustwait for the slowest GPU (e.g. GPU-B) to finish rendering objects 0-3through the corresponding graphics pipeline before moving to the nextphase of rendering.

Geometry pretesting is not performed in rendering timing diagram 700C-1.As such, each of the GPUs must process each of the objects through thecorresponding graphics pipeline. A GPU may not fully render an objectthrough the graphics pipeline if there are no pixels to be drawn for theobject in any region assigned (e.g. in a corresponding set) to thecorresponding GPU for object rendering. For example, when an object doesnot overlap, only the geometry processing stage of the graphics pipelineis executed. However, this still takes some time for processing.

In particular, GPU-A does not fully render objects 0, 1, and 3, becausethey do not overlap any screen regions (e.g. in a corresponding set)assigned to GPU-A for object rendering. The rendering of these threeobjects is shown in boxes with dashed lines indicating that at least thegeometry processing stage is performed, but the graphics pipeline is notfully performed. GPU-A fully renders object 2 because that objectoverlaps at least one screen region assigned to GPU-A for rendering. Therendering of object 2 is shown in a box with solid lines indicating thatall of the stages of the corresponding graphics pipeline are performed.Similarly, GPU-B does not fully render object 1 (shown with a box withdashed lines) (i.e. performing at least geometry processing stage), butfully renders objects 0, 2, and 3 (shown with boxes with solid lines)because those objects overlap at least one screen region (e.g. in acorresponding set) assigned to GPU-B for rendering. Also, GPU-C does notfully render objects 0 and 2 (shown with boxes with dashed lines) (i.e.performing at least geometry processing stage), but fully rendersobjects (shown with boxes with solid lines) because those objectsoverlap at least one screen region (e.g. in a corresponding set)assigned to GPU-C for rendering. Further, GPU-D does not fully renderobject 0 (shown with a box with dashed lines) (i.e. performing at leastgeometry processing stage), but fully renders objects 1, 2, and 3 (shownwith boxes with solid lines) because those objects overlap at least onescreen region (e.g. in a corresponding set) assigned to GPU-D forrendering.

Rendering timing diagram 700C-2 shows geometry pretesting 701′ andrendering 702′ of objects 0-3 using multiple GPUs. Vertical line 750 aindicates the start of the phase of rendering (e.g. including geometrypretesting and rendering) for the objects, and vertical line 750 b showsthe end of the phase of rendering for the objects in rendering timingdiagram 700C-2. The start and end points along timeline 740 for thephase of rendering shown in timing diagram 700C-2 representsynchronization points, wherein each of the four GPUs are synchronizedwhen executing a corresponding GPU pipeline, as previously described.For instance, at vertical line 750 b indicating the end of the phase ofrendering, all GPUs must wait for the slowest GPU (e.g. GPU-B) to finishrendering objects 0-3 through the corresponding graphics pipeline beforemoving to the next phase of rendering.

First, geometry pretesting 701′ is performed by the GPUs, wherein eachGPU performs geometry pretesting for a subset of the geometry of theimage frame against all the screen regions, wherein each screen regionis assigned to a corresponding GPU for object rendering. As previouslydescribed, each of the GPUs is assigned to a corresponding portion ofthe geometry associated with the image frame. Geometry pretestinggenerates information about how a particular piece of geometry relatesto each of the screen regions, such as whether or not a piece ofgeometry overlaps any screen regions (e.g. in a corresponding set)assigned to a corresponding GPU for object rendering. That informationis shared with each of the GPUs used for rendering the image frame. Forexample, geometry pretesting 701′ shown in FIG. 7C includes having GPU-Aperform geometry pretesting for object 0, having GPU-B perform geometrypretesting for object 1, having GPU-C perform geometry pretesting forobject 2, and having GPU-D perform geometry pretesting for object 3.Depending on the object being tested, the time for performing geometrypretesting may vary. For example, geometry pretesting of object 0 takesless time than to perform geometry pretesting on object 1. This may bedue to object sizing, the number of screen regions that are overlapped,etc.

After geometry pretesting, each GPU performs rendering for all objectsor pieces of geometry that intersect their screen regions. In oneembodiment, each GPU begins the rendering of its pieces of geometry assoon as geometry testing is finished. That is, there is nosynchronization point between the geometry testing and the rendering.This is possible because the geometry testing information beinggenerated is treated as a hint rather than a hard dependency. Forexample, GPU-A begins rendering object 2 before GPU-B has finishedgeometry pretesting object 1, and as such before GPU-B begins renderingobjects 0, 2, and 3.

Vertical line 750 a is aligned with vertical line 755 a, such that eachof the rendering timing diagrams 700C-1 and 700C-2 begin at the sametime to render objects 0-3. However, the rendering of objects 0-3 shownin rendering timing diagram 700C-2 is performed in less time than therendering shown in rendering timing diagram 700C-1. That is, verticalline 750 b indicating the end of phase of rendering for the lower timingdiagram 700C-2 occurs earlier than the end of phase of rendering for theupper timing diagram 700C-1 as indicated by vertical line 755 b.Specifically, a speed increase 745 when rendering objects 0-3 isrealized when performing multi-GPU rendering of geometry of an image foran application including pretesting geometry against screen regionsbefore rendering, and providing the results of the geometry pretestingas information (e.g. hints). As shown, speed increase 745 is the timedifference between vertical line 750 b of timing diagram 700C-2 andvertical line 755 b of timing diagram 700C-1.

The speed increase is realized through the generation and sharing ofinformation generated during geometry pretesting. For example, duringgeometry pretesting GPU-A generates information indicating that object 0need only be rendered by GPU-B. As such, GPU-B is informed that itshould render object 0, and the other GPUs (e.g. GPU-A, GPU-C, andGPU-D) may skip rendering of object 0 entirely, as object 0 does notoverlap any regions (e.g. in corresponding sets) assigned to those GPUsfor object rendering. For example, these GPUs need not perform thegeometry processing stage, whereas without geometry pretesting thisstage was processed even though these GPUs would not fully render object0, as is shown in timing diagram 700C-1. Also, during geometrypretesting GPU-B generates information indicating that object 1 shouldbe rendered by GPU-C and GPU-D, and that GPU-A and GPU-B may skiprendering of object 1 entirely, as object 1 does not overlap any region(e.g. in respective corresponding sets) assigned to GPU-A or GPU-B forobject rendering. Also, during geometry pretesting GPU-C generatesinformation indicating that object 2 should be rendered by GPU-A, GPU-B,and GPU-D, and that GPU-C may skip rendering of object 2 entirely, asobject 2 does not overlap any region (e.g. in a corresponding set)assigned to CPU-C for object rendering. Further, during geometrypretesting GPU-D generates information indicating that object 3 shouldbe rendered by GPU-B, GPU-C, and GPU-D, and that GPU-A may skiprendering of object 3 entirely, as object 3 does not overlap any region(e.g. in a corresponding set) assigned to GPU-A for object rendering.

Because the information generated from geometry pretesting is sharedbetween the GPUs, each GPU can determine which objects to render. Assuch, after geometry pretesting is performed and results from thetesting is shared with all the GPUs, then each GPU has information withregards to which objects or pieces of geometry need to be rendered bythe corresponding GPU. For example, GPU-A renders object 2; GPU-Brenders objects 0, 2, and 3; GPU-C renders objects 1 and 3; and GPU-Drenders objects 1, 2, and 3.

In particular, GPU A performs geometry processing for object 1, anddetermines that object 1 can be skipped by GPU-B, as object 1 does notoverlap any region (e.g. in a corresponding set) assigned to GPU-B forobject rendering. In addition, object 1 is not fully rendered by GPU-A,as it does not overlap any region (e.g. in a corresponding set) assignedto GPU-A for object rendering. Since the determination that there is nooverlap of object 1 by any region assigned to GPU-B is made before GPU-Bbegins geometry processing for object 1, GPU-B skips the rendering ofobject 1.

FIGS. 8A-8B show object testing against screen regions 820A and 820B,wherein the screen regions may be interleaved (e.g. screen regions 820Aand 820B show a portion of a display). In particular, multi-GPUrendering of objects is performed for a single image frame, or each ofone or more image frames in a sequence of image frames by performinggeometry testing before rendering objects in the screen. As shown, GPU-Ais assigned responsibility for rendering objects in screen region 820A.GPU-B is assigned responsibility for rendering objects in screen region820B. Information is generated for “pieces of geometry,” wherein thepieces of geometry can be an entire object or portions of objects. Forexample, a piece of geometry can be an object 810, or portions of object810.

FIG. 8A illustrates object testing against screen regions when multipleGPUs collaborate to render a single image, in accordance with oneembodiment of the present disclosure. As previously described, thepieces of geometry can be objects, such that the pieces correspond tothe geometry used by or generated by a corresponding draw call. Duringgeometry pretesting, object 810 may be determined to overlap region820A. That is, portion 810A of object 810 overlaps region 820A. In thatcase, GPU-A is tasked to render object 810. Also, during geometrypretesting object 810 may be determined to overlap region 820B. That is,portion 810B of object 810 overlaps region 820B. In that case, GPU-B isalso tasked to render object 810.

FIG. 8B illustrates testing of portions of an object against screenregions and/or screen sub-regions when multiple GPUs collaborate torender a single image frame, in accordance with one embodiment of thepresent disclosure. That is, the pieces of geometry can be portions ofobjects. For example, object 810 may be split into pieces, such that thegeometry used by or generated by a draw call is subdivided into smallerpieces of geometry. In one embodiment, the pieces of geometry are eachroughly the size for which the position cache and/or parameter cache areallocated. In that case, the information (e.g. hint or hints) aregenerated for those smaller pieces of geometry during geometry testing,wherein the information is used by the rendering GPU, as previouslydescribed.

For example, object 810 is split into smaller objects, such that thepieces of geometry used for region testing corresponds to these smallerobjects. As shown, object 810 is split into pieces of geometry “a”, “b”,“c”, “d”, “e”, and “f”. After geometry pretesting, GPU-A renders onlypieces of geometry “a”, “b”, “c”, “d”, and “e”. That is, GPU-A can skiprendering piece of geometry “f”. Also, after geometry pretesting, GPU-Brenders only pieces of geometry “d,” “e”, and “f” That is, GPU-B canskip rendering pieces of geometry “a”, “b”, and “c”.

In one embodiment, as the geometry processing stage is configured toperform both vertex processing and primitive processing, it is possibleto perform geometry pretesting on a piece of geometry using the shadersin the geometry processing stage. For example, the geometry processingstage generates the information (e.g. hint), such as by testing abounding frustum for the geometry against GPU screen regions, that maybe performed by software shader operations. In one embodiment, this testis accelerated through the use of a dedicated instruction orinstructions implemented through hardware, thereby implementing asoftware/hardware solution. That is, the dedicated instruction orinstructions is used to accelerate the generation of the informationregarding the piece of geometry and its relation to screen regions. Forexample, the homogeneous coordinates of the vertices of the primitive ofa piece of geometry are provided as inputs to the instruction forgeometry pretesting in the geometry processing stage. The testing maygenerate a Boolean return value for each GPU that indicates whether ornot the primitive overlaps any screen region (e.g. in a correspondingset) assigned to that GPU for object rendering. As such, the information(e.g. hint) generated during geometry pretesting regarding thecorresponding piece of geometry and its relation to screen regions isgenerated by shaders in the geometry processing stage.

In another embodiment, the geometry pretesting on a piece of geometrycan be performed in a hardware rasterization stage. For example, ahardware scan converter may be configured to perform geometrypretesting, such that the scan converter generates information regardingall the screen regions assigned to the plurality of GPUs for objectrendering of the corresponding image frame.

In still another embodiment, the pieces of geometry can be primitives.That is, the portions of objects used for geometry pretesting may beprimitives. As such, the information generated during geometrypretesting (e.g. hint) by one GPU indicates whether or not individualtriangles (e.g. representing primitives) need to be rendered by anotherrendering GPU.

In one embodiment, the information generated during geometry pretestingand shared by the GPUs used for rendering includes a number ofprimitives (e.g. a surviving primitive count) that overlap any screenregion (e.g. in a corresponding set) that is assigned to a correspondingGPU for object rendering. The information may also include the number ofvertices used for building or defining those primitives. That is, theinformation includes a surviving vertex count. As such, when renderingthe corresponding rendering GPU may use the supplied vertex count toallocate space in the position cache and parameter cache. For example,vertices that are not needed do not have any allocated space, which mayincrease the efficiency of rendering, in one embodiment.

In other embodiments, the information generated during geometrypretesting (e.g. hint) includes the specific primitives (e.g. survivingprimitives as an exact match) that overlap any screen region (e.g. in acorresponding set) assigned to the corresponding GPU for objectrendering. That is, the information generated for the rendering GPUincludes a specific set of primitives for rendering. The information mayalso include the specific vertices used for building or defining thoseprimitives. That is, the information generated for the rendering GPUincludes a specific set of vertices for rendering. This information may,for example, save the other rendering GPU time during its geometryprocessing stage when rendering the piece of geometry.

In still other embodiments, there may be processing overhead (eithersoftware or hardware) associated with generating the information duringgeometry testing. In that case, it may be beneficial to skip generatinginformation for certain pieces of geometry. That is, informationprovided as hints is generated for certain objects but not for others.For example, a piece of geometry (e.g., an object or portions of theobject) that represents a skybox or a large piece of terrain may includetriangles that are large. In that case, it is likely that each GPU usedfor multi-GPU rendering of an image frame or each of one or more imageframes in a sequence of image frames will need to render those pieces ofgeometry. That is, the information may be generated or not generateddepending on the properties of the corresponding piece of geometry.

FIGS. 9A-9C illustrates various strategies for assigning screen regionsto corresponding GPUs when multiple GPUs collaborate to render a singleimage, in accordance with one embodiment of the present disclosure. Toachieve GPU processing efficiency various techniques may be used whendividing the screen into regions, such as increasing or decreasing thenumber of regions (e.g., to choose the correct amount of regions),interleaving regions, increasing or decreasing the number of regions forinterleaving, selecting a particular pattern when interleaving regions,etc. For instance, the multiple GPUs are configured to perform multi-GPUrendering of geometry for an image frame generated by an application bypretesting the geometry against interleaved screen regions beforerendering objects in a corresponding image. The configurations of screenregions in FIGS. 9A-9C are designed to reduce any imbalance of renderingtime between the plurality of GPUs. The complexity of the test (e.g.overlap a corresponding screen region) varies depending on how thescreen regions are assigned to GPUs. As shown in the diagrams shown inFIGS. 9A-9C, the bold box 910 is the outline of a corresponding screenor display used when rendering the image.

In one embodiment, each of the plurality of screen regions or pluralityof regions is uniformly sized. In one embodiment, each of the pluralityof screen regions is not uniform in size. In still another embodiment,the number and sizing screen regions in a plurality of screen regionschanges dynamically.

In particular, FIG. 9A illustrates a straightforward pattern 900A forscreen 910. Each of the screen regions is uniformly sized. For example,the size of each of the regions may be a rectangle of a dimension thatis a power of 2 pixels. For example, each region may be 256×256 pixelsin size. As shown, the region assignment is a checkerboard pattern, withone row of A and B regions alternated with another row of B and Cregions. The pattern 900A may be easily tested during geometrypretesting. However, there may be some rendering inefficiencies. Forexample, the screen area assigned to each GPU is substantially different(i.e., there is less coverage for screen region C and region D in screen910), which may lead to an imbalance in the rendering time for each GPU.

FIG. 9B illustrates pattern 900B of screen regions for screen 910. Eachof the screen or sub regions is uniformly sized. The screen regions areassigned and distributed so as to reduce the imbalance of rendering timebetween the GPUs. For example, assignment of GPUs to screen regions inpattern 900B results in nearly equal amounts of screen pixels assignedto each GPU across screen 910. That is, the screen regions are assignedto GPUs in such a way as to equalize screen area or coverage in screen910. For example, if each region may be 256×256 pixels in size, each ofthe regions have approximately the same coverage in screen 910. Inparticular, the set of screen regions A covers an area 6×256×256 pixelsin size, the set of screen regions B covers an area 5.75×256×256 pixelsin size, the set of screen regions C covers an area 5.5×256×256 pixelsin size, and the set of screen regions D covers an area 5.5×256×256pixels in size.

FIG. 9C illustrates pattern 900C of screen regions for screen 910. Eachof the screen regions is not uniform in size. That is, screen regionsfor which GPUs are assigned responsibility for rendering objects may notbe uniform in size. In particular, screen 910 is divided such that eachGPU is assigned to an identical number of pixels. For example, if a 4Kdisplay (3840×2160) were to be divided equally into four regionsvertically, then each region would be 520 pixels tall. However,typically GPUs perform many operations in 32×32 blocks of pixels, and520 pixels is not a multiple of 32 pixels. As such, pattern 900C mayinclude blocks that are at a height of 512 pixels (a multiple of 32),and other blocks that are at a height of 544 pixels (also a multiple of32), in one embodiment. Other embodiments may use differently sizedblocks. Pattern 900C shows equal amounts of screen pixels assigned toeach GPU, by using non-uniform screen regions.

In still another embodiment, the needs of the application whenperforming rendering of images change over time, and the screen regionsare chosen dynamically. For example, if it is known that most of therendering time is spent on the lower half of the screen, then it wouldbe advantageous to assign regions in such a way that nearly equalamounts of screen pixels in the lower half of the display are assignedto each GPU used for rendering the corresponding image. That is, theregions assigned to each of the GPUs used for rendering thecorresponding image may be changed dynamically. For instance, thechanges may be applied based on game modes, different games, size ofscreen, pattern chosen for the regions, etc.

FIG. 10 is a diagram illustrating various distributions of theassignment of GPUs to pieces of geometry for purposes of performinggeometry pretesting, in accordance with one embodiment of the presentdisclosure. That is, FIG. 10 shows the distribution of responsibilityfor the generation of information during geometry pretesting betweenmultiple GPUs. As previously described, each GPU is assigned to acorresponding portion of the geometry of an image frame, wherein thatportion may be further partitioned into objects, portions of objects,geometry, pieces of geometry, etc. Geometry pretesting includesdetermining whether or not a particular piece of geometry overlaps anyscreen region or screen regions that is assigned to a corresponding GPUfor object rendering. Geometry pretesting is typically performed inembodiments simultaneously for all geometry (e.g. all pieces ofgeometry) of a corresponding image frame by the GPUs. In that manner,geometry testing is performed collaboratively by the GPUs allows eachGPU to know which pieces of geometry to render, and which pieces ofgeometry to skip rendering, as previously described.

As shown in FIG. 10 , each piece of geometry may be an object, portionof an object, etc. For example, the pieces of geometry may be portionsof objects, such as pieces roughly the size at which the position and/orparameter caches are allocated, as previously described. Purely forillustration, object 0 (e.g. as specified to be rendered by commands 722in the rendering command buffer 700A) is split into pieces “a”, “b”,“c”, “d”, “e” and “f”, such as object 810 in FIG. 8B. Also, object 1(e.g. as specified to be rendered by commands 724 in the renderingcommand buffer 700A) is split into pieces “g”, “h”, and “i”. Further,object 2 (e.g. as specified to be rendered by commands 724 in therendering command buffer 700A) is split into pieces “j”, “k”, “1”, “m”,“n”, and “o”. The pieces may be ordered (e.g., a-o) for purposes ofdistributing responsibility for geometry testing to the GPUs.

Distribution 1010 (e.g. the ABCDABCDABCD . . . row) shows an evendistribution of the responsibility for performing geometry testingbetween a plurality of GPUs. In particular, rather than having one GPUtake the first quarter of the geometry (e.g. in a block, such as GPU Atakes the first four pieces of the approximately sixteen total piecesincluding “a”, “b”, “c” and “d” for geometry testing), and the secondGPU take the second quarter, etc., assignment to GPUs is interleaved.That is, successive pieces of geometry are assigned to different GPUs.For example, piece “a” is assigned to GPU-A, piece “b” is assigned toGPU-B, piece “c” is assigned to GPU-C, piece “d” is assigned to GPU-D,piece “e” is assigned to GPU-A, piece “f” is assigned to GPU-B, piece“g” is assigned to GPU-C, etc. As a result, processing of geometrytesting is roughly balanced between the GPUs (e.g., GPU-A, GPU-B, GPU-C,and GPU-D).

Distribution 1020 (e.g., the ABBCDABBCDABBCD . . . row) shows anasymmetric distribution of the responsibility for performing geometrytesting between a plurality of GPUs. The asymmetric distribution may beadvantageous when certain GPUs have more time to perform geometrytesting than other GPUs when rendering a corresponding image frame. Forexample, one GPU may have finished rendering objects for the previousframe or frames of a scene earlier than the other GPUs, and therefore(since it is anticipated it will finish earlier this frame as well) itcan be assigned more pieces of geometry for performing geometry testing.Again, the assignment to GPUs is interleaved. As shown, GPU-B isassigned more pieces of geometry for geometry pretesting than otherGPUs. For illustration, piece “a” is assigned to GPU-A, piece “b” isassigned to GPU-B, piece “c” is also assigned to GPU-B, piece “d” isassigned to GPU-C, piece “e” is assigned to GPU-D, piece “f” is assignedto GPU-A, piece “g” is assigned to GPU-B, piece “h” is also assigned toGPU-B, piece “i” is assigned to GPU-C, etc. Though the assignment ofgeometry testing to GPUs may not be balanced, the combined processing ofthe complete phase (e.g. geometry pretesting and rendering of geometry)may turn out to be roughly balanced (e.g. each GPU spends approximatelythe same amount of time to perform geometry pretesting and rendering ofgeometry).

FIGS. 11A-11B illustrate the use of statistics for one or more imageframes when assigning responsibility for performing geometry testingbetween a plurality of GPUs. For example, based on statistics some GPUsmay process more or fewer pieces of geometry during geometry testing togenerate information useful when rendering.

In particular, FIG. 11A is a diagram illustrating the pretesting andrendering of geometry of a previous image frame by a plurality of GPUs,and the use of statistics collected during rendering to influence theassignment of pretesting of geometry of a current image frame to theplurality of GPUs in the current image frame, in accordance with oneembodiment of the present disclosure. Purely for illustration, in thesecond frame 1100B of FIG. 11A, GPU-B processes twice as many pieces ofgeometry (e.g. during pretesting) than the other GPUs (e.g. GPU-A,GPU-C, and GPU-D). The distribution and assignment of more pieces ofgeometry to GPU-B to perform geometry pretesting in a current imageframe is based on statistics collected during rendering of the previousimage frame, or previous image frames.

For example, timing diagram 1100A shows geometry pretesting 701A andrendering 702A for a previous image frame, wherein four GPUs (e.g.GPU-A, GPU-B, GPU-C, and GPU-D) are used for both processes. Theassignment of the geometry (e.g. pieces of geometry) of the previousimage frame is evenly distributed between the GPUs. This is shown by theroughly balanced performances of geometry pretesting 701A by each of theGPUs.

Rendering statistics collected from one or more image frames may be usedin determining how to perform geometry testing and rendering of acurrent image frame. That is, the statistics may be provided asinformation for use when performing geometry testing and rendering of asubsequent image frame (e.g. the current image frame). For example,statistics collected during rendering of the objects (e.g. pieces ofgeometry) of the previous image frame may indicate that GPU-B hasfinished rendering earlier than the other GPUs. In particular, GPU-B hasidle time 1130A after rendering its portion of the geometry thatoverlaps any screen region (e.g. in a corresponding set) assigned toGPU-B for object rendering. Each of the other GPU-A, GPU-C, and GPU-Dperform rendering approximately up to the end 710 of the correspondingframe period of the previous image frame.

The previous image frame and the current image frame may be generatedfor a particular scene when executing an application. As such, theobjects from scene to scene may be approximately similar in number andlocation. In that case, the time for performing geometry pretesting andrendering would be similar for GPUs between multiple image frames in asequence of image frames. That is, it is reasonable to presume thatGPU-B will also have idle time when performing geometry testing andrendering in the current image frame, based on the statistics. As such,GPU-B may be assigned more pieces of geometry for geometry pretesting inthe current frame. For example, by having GPU-B process more pieces ofgeometry during geometry pretesting, the result is that GPU-B finishesat approximately the same time as the other GPUs after rendering objectsin the current image frame. That is, each of the GPU-A, GPU-B, GPU-C,and GPU-D perform rendering approximately up to the end 711 of thecorresponding frame period of the current image frame. In oneembodiment, the total time to render the current image frame is reduced,such that it takes less time to render the current image frame whenusing rendering statistics. As such, statistics for the rendering of theprevious frame and/or previous frames may be used to tune the geometrypretesting, such as the distribution of the assignment of the geometry(e.g. pieces of geometry) between the GPUs in the current image frame.

FIG. 11B is a flow diagram 1100B illustrating a method for graphicsprocessing including pretesting and rendering of geometry of a previousimage frame by a plurality of GPUs, and the use of statistics collectedduring rendering to influence the assignment of pretesting of geometryof a current image frame to the plurality of GPUs in the current imageframe, in accordance with one embodiment of the present disclosure. Thediagram of FIG. 11A illustrates the use of statistics in the method offlow diagram 1100B to determine the distribution of assignments ofgeometry (e.g. pieces of geometry) between the GPUs for an image frame.As previously described, various architectures may include multiple GPUscollaborating to render a single image by performing multi-GPU renderingof geometry for an application, such as within one or more cloud gamingservers of a cloud gaming system, or within a stand-alone system, suchas a personal computer or gaming console that includes a high-endgraphics card having multiple GPUs, etc.

In particular, at 1110 the method includes rendering graphics for anapplication using a plurality of GPUs, as previously described. At 1120,the method includes dividing responsibility for rendering geometry ofthe graphics between the plurality of GPUs based on a plurality ofscreen regions. Each GPU has a corresponding division of theresponsibility which is known to the plurality of GPUs. Morespecifically, each of the GPUs is responsible for rendering geometry ina corresponding set of screen regions of the plurality of screenregions, wherein the corresponding set of screen regions includes one ormore screen regions, as previously described. In one embodiment, thescreen regions are interleaved (e.g. when a display is divided into setsof screen regions for geometry pretesting and rendering).

At 1130, the method includes rendering a first plurality of pieces ofgeometry at the plurality of GPUs of a previous image frame generated byan application. For example, timing diagram 1100A illustrates the timingof performing geometry testing of pieces of geometry and rendering ofobjects (e.g. pieces of geometry) in the previous image frame. At 1140,the method includes generating statistics for the rendering of theprevious image frame. That is, statistics may be collected whenrendering the previous image frame.

At 1150, the method includes assigning based on the statistics a secondplurality of pieces of geometry of a current image frame generated bythe application to the plurality of GPUs for geometry testing. That is,those statistics may be used to assign the same, fewer, or more piecesof geometry for geometry testing to a particular GPU when rendering thenext, or current image frame. In some cases, the statistics may indicatethat the pieces in the second plurality of pieces of geometry should beassigned evenly to the plurality of GPUs when performing geometrytesting.

In other cases, the statistics may indicate that the pieces in thesecond plurality of pieces of geometry should be assigned unevenly tothe plurality of GPUs when performing geometry testing. For example, asshown in timeline 1100A statistics may indicate that GPU-B finishesrendering before any of the other GPUs in the previous image frame. Inparticular, it may be determined that a first GPU (e.g. GPU-B) finishedrendering the first plurality of pieces of geometry before a second GPU(e.g. GPU-A) finished rendering the first plurality of pieces ofgeometry (e.g., its portion of pieces of geometry). As previouslydescribed, the first GPU (e.g. GPU-B) renders one or more pieces of thefirst plurality of pieces of geometry that overlap any screen regionassigned to the first GPU for object rendering, and the second GPU (e.g.GPU-A) renders one or more pieces of the first plurality of pieces ofgeometry that overlap any screen region assigned to the second GPU forobject rendering. As such, because it is anticipated based on thestatistics that the first GPU (e.g. GPU-B) will require less time forrendering the second plurality of pieces of geometry than the second GPU(e.g. GPU-A), more pieces of geometry may be assigned to the first GPUfor geometry pretesting when rendering the current image frame. Forexample, a first number of the second plurality of pieces of geometrymay be assigned to the first GPU (e.g. GPU-B) for geometry testing, anda second number of the second plurality of pieces of geometry may beassigned to the second GPU (e.g., GPU-A) for geometry testing, whereinthe first number is higher than the second number (if the time imbalanceis sufficiently large, then GPU-A may be assigned no pieces at all). Inthat manner, GPU-B processes more pieces of geometry than GPU-A duringgeometry testing. For example, timing diagram 1100B shows that GPU-B hasbeen assigned more pieces of geometry, and spends more time performinggeometry testing than the other GPUs.

At 1160, the method includes performing geometry pretesting at a currentimage frame on the second plurality of pieces of geometry to generateinformation regarding each piece of the second plurality of pieces ofgeometry and its relation to each of the plurality of screen regions.The geometry pretesting is performed at each of the plurality of GPUsbased on the assigning. Geometry pretesting is performed at a pretestGPU on a plurality of pieces of geometry of an image frame generated byan application in order to generate information regarding each piece ofgeometry and its relation to each of the plurality of screen regions.

At 1170, the method includes using the information generated for each ofthe second plurality of pieces of geometry to render the plurality ofpieces of geometry during a rendering phase (e.g. to include fullyrendering a piece of geometry or skipping the rendering of that piece ofgeometry at a corresponding GPU). Rendering is typically performed inembodiments simultaneously at each of the GPUs. In particular, theplurality of pieces of geometry of the current image frame is renderedat each of the plurality of GPUs using the information generated foreach of the pieces of geometry.

In other embodiments, the distribution of pieces of geometry to GPUs forgeneration of the information is dynamically adjusted. That is, anassignment of pieces of geometry for a current image frame forperforming geometry pretesting may be dynamically adjusted during therendering of the current image frame. For example, in the example oftiming diagram 1100B, it may be determined that GPU-A was performinggeometry pretesting of its assigned pieces of geometry at a rate slowerthan expected. As such, the pieces of geometry assigned to GPU-A forgeometry pretesting can be reassigned on-the-fly, such as reassigning apiece of geometry from GPU-A to GPU-B, such that GPU-B is now tasked toperform geometry pretesting on that piece of geometry, during the frameperiod used for rendering the current image frame.

FIGS. 12A-12B illustrate another strategy for processing renderingcommand buffers. Previously, one strategy was described in relation toFIG. 7A-7C, wherein a command buffer contains commands for geometrypretesting on objects (e.g. pieces of geometry), followed by commandsfor rendering of the objects (e.g., pieces of geometry). FIGS. 12A-12Bshow a geometry pretesting and rendering strategy that uses shaders thatare capable of performing either operation depending on GPUconfiguration.

In particular, FIG. 12A is a diagram illustrating the use of shadersconfigured to perform both pretesting and rendering of geometry of animage frame in two passes through a portion of the command buffer 1200A,in accordance with one embodiment of the present disclosure. That is,the shaders used for performing commands in the command buffer 1200A maybe configured to perform either geometry pretesting when properlyconfigured, or to perform rendering when properly configured.

As shown, the portion of the command buffer 1200A shown in FIG. 12A isexecuted twice, with different actions resulting from each execution;the first execution results in performing geometry pretesting, and thesecond execution results in performing rendering of the geometry. Thiscan be accomplished in a variety of ways, e.g. the portion of thecommand buffer depicted in 1200A can be explicitly called twice as asubroutine, with different state (e.g. a register setting or value inRAM) explicitly set to different values prior to each call.Alternatively, the portion of the command buffer depicted in 1200A canbe implicitly executed twice, e.g. by using special commands to markbeginning and end of the portion to execute twice, and to implicitly seta different configuration (e.g. a register setting) for the first andsecond executions of the portion of the command buffer. When thecommands in the portion of the command buffer 1200A are executed (e.g.,commands that set state or commands that execute a shader), based on GPUstate, the results of the commands are different (e.g. result inperforming geometry pretesting vs. performing rendering). That is, thecommands in the command buffer 1200A may be configured for geometrypretesting or rendering. In particular, the portion of command buffer1200A includes commands for configuring a state of the one or more GPUsexecuting commands from the rendering command buffer 1200A, and commandsfor executing a shader that performs either geometry pretesting orrendering depending on the state. For example, commands 1210, 1212,1214, and 1216 are each used for configuring a state of the one or moreGPUs for purposes of executing a shader that performs either geometrypretesting or rendering depending on the state. As shown, command 1210configures GPU state so that shader 0 may be executed via commands 1211and perform either geometry pretesting or rendering. Also, command 1212configures GPU state so that shader 1 may be executed via commands 1213to perform geometry pretesting or rendering. In addition, command 1214configures GPU state so that shader 2 may be executed via commands 1215to perform either geometry pretesting or rendering. Finally, command1216 configures GPU state so that shader 3 may be may be executed viacommands 1217 to perform either geometry pretesting or rendering.

On the first traversal 1291 through the command buffer 1200A, based onGPU state set explicitly or implicitly as described above, as well asGPU state configured by commands 1210, 1212, 1214, and 1216, thecorresponding shaders perform geometry pretesting. For example, shader 0is configured to perform geometry pretesting on object 0 (e.g. a pieceof geometry) (e.g. based on the objects shown in FIG. 7B-1 ), shader 1is configured to perform geometry pretesting on object 1, shader 2 isconfigured to perform geometry pretesting on object 2, and shader 3 isconfigured to perform geometry pretesting on object 3.

In one embodiment, based on the GPU state, commands may be skipped orinterpreted differently. For example, certain commands that set state(portions of 1210, 1212, 1214 and 1216) may be skipped based on GPUstate that is set explicitly or implicitly as described above; e.g. ifconfiguring the shader 0 executed via command 1210 requires less GPUstate to be configured for geometry pretesting than when it isconfigured for rendering of geometry, then it may be beneficial to skipsetting the unnecessary portions of the GPU state as setting of GPUstate may carry an overhead. To give another example, certain commandsthat set state (portions of 1210, 1212, 1214 and 1216) may beinterpreted differently based on GPU state that is set explicitly orimplicitly as described above; e.g. if shader 0 executed via command1210 requires different GPU state to be configured for geometrypretesting than when it is configured for rendering of geometry, or ifshader 0 executed via command 1210 requires an input that is differentfor geometry pretesting and for rendering of geometry.

In one embodiment, the shaders configured for geometry pretesting do notallocate space in the position and parameter caches, as previouslydescribed. In another embodiment, a single shader is used to performeither the pretesting or the rendering. This could be done in a varietyof ways, such as via external hardware state that the shader could check(e.g. as set explicitly or implicitly as described above), or via aninput to the shader (e.g. as set by a command that is interpreteddifferently in the first and second passes through the command buffer).

On the second traversal 1292 through the command buffer 1200A, based onGPU state set explicitly or implicitly as described above, as well asGPU state as configured by commands 1210, 1212, 1214, and 1216, thecorresponding shaders perform rendering of pieces of geometry for acorresponding image frame. For example, shader 0 is configured toperform rendering of object 0 (e.g. a piece of geometry) (e.g. based onthe objects shown in FIG. 7B-1 ). Also, shader 1 is configured toperform rendering of object 1, shader 2 is configured to performrendering of object 2, and shader 3 is configured to perform renderingof object 3.

FIG. 12B is a flow diagram 1200B illustrating a method for graphicsprocessing including performing both pretesting and rendering ofgeometry of an image frame using the same set of shaders in two passesthrough a portion of the command buffer, in accordance with oneembodiment of the present disclosure. As previously described, variousarchitectures may include multiple GPUs collaborating to render a singleimage by performing multi-GPU rendering of geometry for an application,such as within one or more cloud gaming servers of a cloud gamingsystem, or within a stand-alone system, such as a personal computer orgaming console that includes a high-end graphics card having multipleGPUs, etc.

In particular, at 1210 the method includes rendering graphics for anapplication using a plurality of GPUs, as previously described. At 1220,the method includes dividing responsibility for rendering geometry ofthe graphics between the plurality of GPUs based on a plurality ofscreen regions. Each GPU has a corresponding division of theresponsibility which is known to the plurality of GPUs. Morespecifically, each of the GPUs is responsible for rendering geometry ina corresponding set of screen regions of the plurality of screenregions, wherein the corresponding set of screen regions includes one ormore screen regions, as previously described. In one embodiment, thescreen regions are interleaved (e.g. when a display is divided into setsof screen regions for geometry pretesting and rendering).

At 1230, the method includes assigning a plurality of pieces of geometryof an image frame to the plurality of GPUs for geometry testing. Inparticular, each of the plurality of GPUs is assigned to a correspondingportion of the geometry associated with an image frame for purpose ofgeometry testing. As previously described, the assignments of pieces ofgeometry may be evenly or unevenly distributed, wherein each portionincludes one or more pieces of geometry, or potentially no pieces ofgeometry at all, in embodiments.

At 1240, the method includes loading first GPU state configuring one ormore shaders to perform the geometry pretesting. For example, dependingon GPU state, a corresponding shader may be configured to performdifferent operations. As such, the first GPU state configurescorresponding shaders to perform geometry pretesting. In the example ofFIG. 12A, this can be set in a variety of ways, e.g. by explicitly orimplicitly setting state externally to the portion of the command bufferdepicted in 1200A, as described above. In particular, the GPU state maybe set in a variety of ways. For example, the CPU or GPU could set avalue in random access memory (RAM), wherein the GPU would check thevalue in RAM. In another example, the state could be internal to theGPU, such as when a command buffer is called as a subroutine twice withinternal GPU state being different between the two subroutine calls.Alternatively, the commands 1210 in FIG. 12A can be interpreteddifferently or skipped based on the state set explicitly or implicitlyas described above. Based on this first GPU state, shader 0 executed bycommand 1211 is configured to perform geometry pretesting.

At 1250, the method includes performing geometry pretesting at theplurality of GPUs on the plurality of pieces of geometry to generateinformation regarding each piece of geometry and its relation to each ofthe plurality of screen regions. As previously described, geometrypretesting may determine whether a piece of geometry overlaps any screenregions (e.g. in a corresponding set) that are assigned to acorresponding GPU for object rendering. Because geometry pretesting istypically performed in embodiments simultaneously for all geometry of acorresponding image frame by the GPUs, each GPU is able to know whichpieces of geometry to render, and which pieces of geometry to skip. Thisends the first traversal through the command buffer, wherein shaders maybe configured to perform each of geometry pretesting and/or rendering,depending on GPU state.

At 1260, the method includes loading second GPU state configuring theone or more shaders to perform rendering. As previously described,depending on GPU state, a corresponding shader may be configured toperform different operations. As such, the second GPU state configurescorresponding shaders (the same shaders previously used to performgeometry pretesting) to perform rendering. In the example of FIG. 12A,based on this second GPU state, shader 0 executed by command 1211 isconfigured to perform rendering.

At 1270, the method includes at each of the plurality of GPUs using theinformation generated for each of the plurality of pieces of geometrywhen rendering the plurality of pieces of geometry (e.g. to includefully rendering a piece of geometry or skipping the rendering of thatpiece of geometry at a corresponding GPU). As previously described, theinformation may indicate whether a piece of geometry overlaps any screenregion (e.g. in a corresponding set) that are assigned to acorresponding GPU for object rendering. That information may be used forrendering each of the plurality of pieces of geometry at each of theplurality of GPUs, such that each GPU can efficiently render only piecesof geometry that overlap at least one screen (e.g. in a correspondingset) assigned to that corresponding GPU for object rendering. This endsthe second traversal through the command buffer, wherein shaders may beconfigured to perform each of geometry pretesting and/or rendering,depending on GPU state.

FIGS. 13A-13B illustrate another strategy for processing renderingcommand buffers. Previously, one strategy was described in relation toFIG. 7A-7C, wherein a command buffer contains commands for geometrypretesting of objects (e.g. pieces of geometry), followed by commandsfor rendering of the objects (e.g., pieces of geometry), and anotherstrategy was described in FIGS. 12A-12B that uses shaders that arecapable of performing either operation depending on GPU configuration.FIGS. 13A-13B show a geometry testing and rendering strategy that usesshaders capable of performing either geometry pretesting or rendering,and wherein the processes of geometry pretesting and rendering areinterleaved for different sets of pieces of geometry, in accordance withembodiments of the present disclosure.

In particular, FIG. 13A is a diagram illustrating the use of shadersconfigured to perform both geometry pretesting and rendering, whereingeometry pretesting and rendering performed for different sets of piecesof geometry are interleaved using separate portions of a correspondingcommand buffer 1300A, in accordance with one embodiment of the presentdisclosure. That is, rather than executing the portion of the commandbuffer 1300A start to finish, the command buffer 1300A is dynamicallyconfigured and executed, so that geometry pretesting and rendering areinterleaved for different sets of pieces of geometry. For example, in acommand buffer, some shaders (e.g. executed via commands 1311 and 1313)are configured for performing geometry pretesting on a first set ofpieces of geometry, wherein after geometry testing is performed thosesame shaders (e.g. executed by commands 1311 and 1313) are thenconfigured for performing rendering. After rendering is performed on thefirst set of pieces of geometry, other shaders (e.g. executed viacommands 1315 and 1317) in the command buffer are configured forperforming geometry pretesting on a second set of pieces of geometry,wherein after geometry pretesting is performed those same shaders (e.g.executed via commands 1315 and 1317) are then configured for performingrendering, and rendering is performed using those commands on the secondset of pieces of geometry. The benefit of this strategy is thatimbalance between GPUs can be addressed dynamically, such as by usingasymmetric interleaving of geometry testing throughout the rendering. Anexample of asymmetric interleaving of geometry testing was previouslyintroduced in distribution 102 of FIG. 10 .

As interleaving of geometry pretesting and rendering occurs dynamically,the configuration (e.g. via a register setting or a value in RAM) of theGPU occurs implicitly, which is to say that an aspect of the GPUconfiguration happens externally to the command buffer. For example, aGPU register may be set to 0 (indicating that geometry pretesting shouldoccur) or 1 (indicating that rendering should occur); the interleavedtraversal of the command buffer and the setting of this register may becontrolled by the GPU based on numbers of objects processed, primitivesprocessed, imbalance between the GPUs, etc. Alternatively, a value inRAM could be used. As a result of this external configuration (meaningset externally to the command buffer), when the commands in the portionof the command buffer 1300A are executed (e.g., commands that set stateor commands that execute a shader), based on GPU state, the results ofthe commands are different (e.g. result in performing geometrypretesting vs. performing rendering). That is, the commands in thecommand buffer 1300A may be configured for geometry pretesting 1391 orrendering 1392. In particular, the portion of the command buffer 1300Aincludes commands for configuring a state of the one or more GPUsexecuting commands from rendering command buffer 1300A, and commands forexecuting a shader that performs either geometry pretesting or renderingdepending on the state. For example, commands 1310, 1312, 1314, and 1316are each used for configuring a state of GPUs for purposes of executinga shader that performs either geometry pretesting or rendering dependingon the state. As shown, command buffer 1310 configures GPU state so thatshader 0 may be executed via commands 1311 either for performinggeometry pretesting or rendering of object 0. Also, command buffer 1312configures GPU state so that shader 1 may be executed via commands 1313either for performing geometry pretesting or rendering of object 1.Also, command buffer 1314 configures GPU state so that shader 2 may beexecuted via commands 1315 either for performing geometry pretesting orrendering of object 2. Further, command buffer 1316 configures GPU stateso that shader 3 may be executed via commands 1317 either for performinggeometry pretesting or rendering of object 3.

Geometry pretesting and rendering may be interleaved for different setsof pieces of geometry. For illustration purposes only, command buffer1300A may be configured to perform geometry pretesting and rendering ofobjects 0 and 1 first, and then command buffer 1300A is configured toperform geometry pretesting and rendering of objects 2 and 3 second. Itis understood that different numbers of pieces of geometry may beinterleaved in different sections. For example, section 1 shows a firsttraversal through command buffer 1300A. Based on GPU state setimplicitly as described above, as well as GPU state as configured bycommands 1310 and 1312, the corresponding shaders perform geometrypretesting. For example, shader 0 is configured to perform geometrypretesting on object 0 (e.g. a piece of geometry) (e.g. based on objectsshown in FIGS. 7B-1 ), and shader 1 is configured to perform geometrypretesting on object 1. Section 2 shows a second traversal throughcommand buffer 1300A. Based on GPU state set implicitly as describedabove, as well as GPU state as configured by commands 1310 and 1312, thecorresponding shaders perform rendering. For example, shader 0 isconfigured to now perform rendering of object 0, and shader 1 isconfigured to now perform rendering of object 1.

Interleaving of the performance of geometry pretesting and rendering ondifferent sets of pieces of geometry is shown in FIG. 13A. Inparticular, section 3 shows a third partial traversal through commandbuffer 1300A. Based on GPU state set implicitly as described above, aswell as GPU state as configured by commands 1314 and 1316, thecorresponding shaders perform geometry pretesting. For example, shader 2(executed via commands 1315) performs geometry testing on object 2 (e.g.a piece of geometry) (e.g. based on objects shown in FIGS. 7B-1 ), andshader 3 (executed via commands 1317) performs geometry testing onobject 3. Section 4 shows a fourth partial traversal through commandbuffer 1300A. Based on GPU state set implicitly as described above, aswell as GPU state as configured by commands 1314 and 1316, thecorresponding shaders perform rendering. For example, shader 2 (executedvia commands 1315) performs rendering of object 2, and shader 3(executed via commands 1317) performs rendering of object 3.

Note that hardware contexts are preserved, or saved and restored. Forexample, the geometry pretesting GPU context at the end of section 1 isneeded at the beginning of section 3 for performing geometry pretesting.Also, the rendering GPU context at the end of section 2 is needed forthe beginning of section 4 for performing rendering.

In one embodiment, based on the GPU state, commands may be skipped orinterpreted differently. For example, certain commands that set state(portions of 1310, 1312, 1314 and 1316) may be skipped based on GPUstate that is set implicitly as described above; e.g. if configuring theshader 0 executed via command 1310 requires less GPU state to beconfigured for geometry testing than when it is configured for renderingof geometry, then it may be beneficial to skip setting the unnecessaryportions of the GPU state as setting of GPU state may carry an overhead.To give another example, certain commands that set state (portions of1310, 1312, 1314 and 1316) may be interpreted differently based on GPUstate that is set implicitly as described above; e.g. if shader 0executed via command 1310 requires different GPU state to be configuredfor geometry testing than when it is configured for rendering ofgeometry, or if shader 0 executed via command 1310 requires an inputthat is different for geometry testing and for rendering of geometry.

In one embodiment, the shaders configured for geometry pretesting do notallocate space in the position and parameter caches, as previouslydescribed. In another embodiment, a single shader is used to performeither the pretesting or the rendering. This could be done in a varietyof ways, such as via external hardware state that the shader could check(e.g. as set implicitly as described above), or via an input to theshader (e.g. as set by a command that is interpreted differently in thefirst and second passes through the command buffer).

FIG. 13B is a flow diagram illustrating a method for graphics processingincluding interleaving pretesting and rendering of geometry of an imageframe for different sets of pieces geometry using separate portions of acorresponding command buffer, in accordance with one embodiment of thepresent disclosure. As previously described, various architectures mayinclude multiple GPUs collaborating to render a single image byperforming multi-GPU rendering of geometry for an application, such aswithin one or more cloud gaming servers of a cloud gaming system, orwithin a stand-alone system, such as a personal computer or gamingconsole that includes a high-end graphics card having multiple GPUs,etc.

In particular, at 1310 the method includes rendering graphics for anapplication using a plurality of GPUs, as previously described. At 1320,the method includes dividing responsibility for rendering geometry ofthe graphics between the plurality of GPUs based on a plurality ofscreen regions. Each GPU has a corresponding division of theresponsibility which is known to the plurality of GPUs. Morespecifically, each of the GPUs is responsible for rendering geometry ina corresponding set of screen regions of the plurality of screenregions, wherein the corresponding set of screen regions includes one ormore screen regions, as previously described. In one embodiment, thescreen regions are interleaved (e.g. when a display is divided into setsof screen regions for geometry pretesting and rendering).

At 1330, the method includes assigning a plurality of pieces of geometryof an image frame to the plurality of GPUs for geometry testing. Inparticular, each of the plurality of GPUs is assigned to a correspondingportion of the geometry associated with an image frame for purpose ofgeometry testing. As previously described, the assignments of pieces ofgeometry may be evenly or unevenly distributed, wherein each portionincludes one or more pieces of geometry, or potentially no pieces ofgeometry at all.

At 1340, the method includes interleaving a first set of shaders in acommand buffer with a second set of shaders, wherein the shaders areconfigured to perform both geometry pretesting and rendering. Inparticular, the first set of shaders is configured to perform geometrypretesting and rendering on a first set of pieces of geometry.Thereafter, the second set of shaders is configured to perform geometrypretesting and rendering on a second set of pieces of geometry. Aspreviously described, geometry pretesting generates correspondinginformation regarding each piece of geometry in the first set or secondset and its relation to each of the plurality of screen regions. Thecorresponding information is used by the plurality of GPUs to rendereach piece of geometry in first set or second set. As previouslydescribed, the GPU state may be set in a variety of ways in order toperform either geometry pretesting or rendering. For example, the CPU orGPU could set a value in random access memory (RAM), wherein the GPUwould check the value in RAM. In another example, the state could beinternal to the GPU, such as when a command buffer is called as asubroutine twice with internal GPU state being different between the twosubroutine calls.

The interleaving process is further described. In particular, the firstset of shaders of a command buffer is configured to perform geometrypretesting on the first set of pieces of geometry, as previouslydescribed. Geometry pretesting is performed at the plurality of GPUs onthe first set of pieces of geometry to generate first informationregarding each piece of geometry in the first set and its relation toeach of the plurality of screen regions. Then, the first set of shadersis configured to perform rendering of the first set of pieces ofgeometry, as previously described. Thereafter, the first information isused when rendering the plurality of pieces of geometry at each of theplurality of GPUs (e.g. to include fully rendering the first set ofpieces of geometry or skipping the rendering of the first set of piecesof geometry at a corresponding GPU). As previously described, theinformation indicates which pieces of geometry overlap screen regionsassigned to a corresponding GPU for object rendering. For example, theinformation may be used to skip rendering a piece of geometry at a GPUwhen that information indicates that the piece of geometry does tooverlap any screen region (e.g. in a corresponding set) assigned to theGPU for object rendering.

The second set of shaders is then used for geometry testing andrendering of the second set of pieces of geometry. In particular, thesecond set of shaders of a command buffer is configured to performgeometry pretesting on the second set of pieces of geometry, aspreviously described. Then, geometry testing is performed at theplurality of GPUs on the second set of pieces of geometry to generatesecond information regarding each piece of geometry in the second setand its relation to each of the plurality of screen regions. Then, thesecond set of shaders is configured to perform rendering of the secondset of pieces of geometry, as previously described. Thereafter,rendering of the second set of pieces of geometry is performed at eachof the plurality of GPUs using the second information. As previouslydescribed, the information indicates which pieces of geometry overlapscreen regions (e.g. of a corresponding set) assigned to a correspondingGPU for object rendering.

Though the above describes the plurality of GPUs as processing thegeometry in lockstep (i.e. the plurality of GPUs performs geometrypretesting, then the plurality of GPUs performs rendering), in someembodiments the GPUs are not explicitly synchronized with each other,e.g. one GPU may be rendering the first set of pieces of geometry whilea second GPU is performing geometry pretesting on the second set ofpieces of geometry.

FIG. 14 illustrates components of an example device 1400 that can beused to perform aspects of the various embodiments of the presentdisclosure. For example, FIG. 14 illustrates an exemplary hardwaresystem suitable for multi-GPU rendering of geometry for an applicationby pretesting geometry against screen regions, which may be interleaved,before rendering objects for an image frame, in accordance withembodiments of the present disclosure. This block diagram illustrates adevice 1400 that can incorporate or can be a personal computer, a servercomputer, gaming console, mobile device, or other digital device, eachof which is suitable for practicing an embodiment of the invention.Device 1400 includes a central processing unit (CPU) 1402 for runningsoftware applications and optionally an operating system. CPU 1402 maybe comprised of one or more homogeneous or heterogeneous processingcores.

In accordance with various embodiments, CPU 1402 is one or moregeneral-purpose microprocessors having one or more processing cores.Further embodiments can be implemented using one or more CPUs withmicroprocessor architectures specifically adapted for highly paralleland computationally intensive applications, such as media andinteractive entertainment applications, of applications configured forgraphics processing during execution of a game.

Memory 1404 stores applications and data for use by the CPU 1402 and GPU1416. Storage 1406 provides non-volatile storage and other computerreadable media for applications and data and may include fixed diskdrives, removable disk drives, flash memory devices, and CD-ROM,DVD-ROM, Blu-ray, HD-DVD, or other optical storage devices, as well assignal transmission and storage media. User input devices 1408communicate user inputs from one or more users to device 1400, examplesof which may include keyboards, mice, joysticks, touch pads, touchscreens, still or video recorders/cameras, and/or microphones. Networkinterface 1409 allows device 1400 to communicate with other computersystems via an electronic communications network, and may include wiredor wireless communication over local area networks and wide areanetworks such as the internet. An audio processor 1412 is adapted togenerate analog or digital audio output from instructions and/or dataprovided by the CPU 1402, memory 1404, and/or storage 1406. Thecomponents of device 1400, including CPU 1402, graphics subsystemincluding GPU 1416, memory 1404, data storage 1406, user input devices1408, network interface 1409, and audio processor 1412 are connected viaone or more data buses 1422.

A graphics subsystem 1414 is further connected with data bus 1422 andthe components of the device 1400. The graphics subsystem 1414 includesat least one graphics processing unit (GPU) 1416 and graphics memory1418. Graphics memory 1418 includes a display memory (e.g. a framebuffer) used for storing pixel data for each pixel of an output image.Graphics memory 1418 can be integrated in the same device as GPU 1416,connected as a separate device with GPU 1416, and/or implemented withinmemory 1404. Pixel data can be provided to graphics memory 1418 directlyfrom the CPU 1402. Alternatively, CPU 1402 provides the GPU 1416 withdata and/or instructions defining the desired output images, from whichthe GPU 1416 generates the pixel data of one or more output images. Thedata and/or instructions defining the desired output images can bestored in memory 1404 and/or graphics memory 1418. In an embodiment, theGPU 1416 includes 3D rendering capabilities for generating pixel datafor output images from instructions and data defining the geometry,lighting, shading, texturing, motion, and/or camera parameters for ascene. The GPU 1416 can further include one or more programmableexecution units capable of executing shader programs.

The graphics subsystem 1414 periodically outputs pixel data for an imagefrom graphics memory 1418 to be displayed on display device 1410, or tobe projected by a projection system (not shown). Display device 1410 canbe any device capable of displaying visual information in response to asignal from the device 1400, including CRT, LCD, plasma, and OLEDdisplays. Device 1400 can provide the display device 1410 with an analogor digital signal, for example.

Other embodiments for optimizing the graphics subsystem 1414 couldinclude multi-GPU rendering of geometry for an application by pretestingthe geometry against screen regions, which may be interleaved, beforerendering objects for an image frame. The graphics subsystem 1414 couldbe configured as one or more processing devices.

For example, the graphics subsystem 1414 may be configured to performmulti-GPU rendering of geometry for an application, wherein multiplegraphics subsystems could be implementing graphics and/or renderingpipelines for a single application, in one embodiment. That is, thegraphics subsystem 1414 includes multiple GPUs used for rendering animage or each of one or more images of a sequence of images whenexecuting an application.

In other embodiments, the graphics subsystem 1414 includes multiple GPUdevices, which are combined to perform graphics processing for a singleapplication that is executing on a corresponding CPU. For example, themultiple GPUs can perform multi-GPU rendering of geometry for anapplication by pretesting the geometry against screen regions, which maybe interleaved, before rendering objects for an image frame. In otherexamples, the multiple GPUs can perform alternate forms of framerendering, wherein GPU 1 renders a first frame, and GPU 2 renders asecond frame, in sequential frame periods, and so on until reaching thelast GPU whereupon the initial GPU renders the next video frame (e.g. ifthere are only two GPUs, then GPU 1 renders the third frame). That isthe GPUs rotate when rendering frames. The rendering operations canoverlap, wherein GPU 2 may begin rendering the second frame before GPU 1finishes rendering the first frame. In another implementation, themultiple GPU devices can be assigned different shader operations in therendering and/or graphics pipeline. A master GPU is performing mainrendering and compositing. For example, in a group including three GPUs,master GPU 1 could perform the main rendering (e.g. a first shaderoperation) and compositing of outputs from slave GPU 2 and slave GPU 3,wherein slave GPU 2 could perform a second shader (e.g. fluid effects,such as a river) operation, the slave GPU 3 could perform a third shader(e.g. particle smoke) operation, wherein master GPU 1 composites theresults from each of GPU 1, GPU 2, and GPU 3. In that manner, differentGPUs can be assigned to perform different shader operations (e.g. flagwaving, wind, smoke generation, fire, etc.) to render a video frame. Instill another embodiment, each of the three GPUs could be assigned todifferent objects and/or parts of a scene corresponding to a videoframe. In the above embodiments and implementations, these operationscould be performed in the same frame period (simultaneously inparallel), or in different frame periods (sequentially in parallel).

Accordingly, the present disclosure describes methods and systemsconfigured for multi-GPU rendering of geometry for an application bypretesting the geometry against screen regions, which may beinterleaved, before rendering of objects for an image frame or each ofone or more image frames in a sequence of image frames when executing anapplication.

It should be understood that the various embodiments defined herein maybe combined or assembled into specific implementations using the variousfeatures disclosed herein. Thus, the examples provided are just somepossible examples, without limitation to the various implementationsthat are possible by combining the various elements to define many moreimplementations. In some examples, some implementations may includefewer elements, without departing from the spirit of the disclosed orequivalent implementations.

Embodiments of the present disclosure may be practiced with variouscomputer system configurations including hand-held devices,microprocessor systems, microprocessor-based or programmable consumerelectronics, minicomputers, mainframe computers and the like.Embodiments of the present disclosure can also be practiced indistributed computing environments where tasks are performed by remoteprocessing devices that are linked through a wire-based or wirelessnetwork.

With the above embodiments in mind, it should be understood thatembodiments of the present disclosure can employ variouscomputer-implemented operations involving data stored in computersystems. These operations are those requiring physical manipulation ofphysical quantities. Any of the operations described herein that formpart of embodiments of the present disclosure are useful machineoperations. Embodiments of the disclosure also relate to a device or anapparatus for performing these operations. The apparatus can bespecially constructed for the required purpose, or the apparatus can bea general-purpose computer selectively activated or configured by acomputer program stored in the computer. In particular, variousgeneral-purpose machines can be used with computer programs written inaccordance with the teachings herein, or it may be more convenient toconstruct a more specialized apparatus to perform the requiredoperations.

The disclosure can also be embodied as computer readable code on acomputer readable medium. The computer readable medium is any datastorage device that can store data, which can be thereafter be read by acomputer system. Examples of the computer readable medium include harddrives, network attached storage (NAS), read-only memory, random-accessmemory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes and other optical andnon-optical data storage devices. The computer readable medium caninclude computer readable tangible medium distributed over anetwork-coupled computer system so that the computer readable code isstored and executed in a distributed fashion.

Although the method operations were described in a specific order, itshould be understood that other housekeeping operations may be performedin between operations, or operations may be adjusted so that they occurat slightly different times, or may be distributed in a system whichallows the occurrence of the processing operations at various intervalsassociated with the processing, as long as the processing of the overlayoperations are performed in the desired way.

Although the foregoing disclosure has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications can be practiced within the scope of theappended claims. Accordingly, the present embodiments are to beconsidered as illustrative and not restrictive, and embodiments of thepresent disclosure is not to be limited to the details given herein, butmay be modified within the scope and equivalents of the appended claims.

What is claimed is:
 1. A method, comprising: rendering a first imageframe generated by an application using a plurality of graphicsprocessing units (GPUs); generating statistics for the rendering of thefirst image frame; assigning a plurality of pieces of geometry of asecond image frame generated by the application to the plurality of GPUsfor geometry testing based on the statistics, wherein the second imageframe follows the first image frame; and performing the geometry testingat the plurality of GPUs on the plurality of pieces of geometry togenerate information regarding each of the plurality of pieces ofgeometry and its relation to each of a plurality of screen regions, thegeometry testing performed at each of the plurality of GPUs based on theassigning.
 2. The method of claim 1, further comprising: generatingstatistics for rendering a plurality of previous image frames, whereineach of the plurality of previous image frames is rendered before thesecond image frame is rendered, wherein the plurality of pieces ofgeometry of the second image frame is assigned to the plurality of GPUsfor geometry testing based on the statistics for the plurality ofprevious image fames that includes the statistics for the rendering ofthe first image frame.
 3. The method of claim 2, wherein the statisticsfor rendering the plurality of previous image frames include timing ofrendering geometry for each of the plurality of image frames by theplurality of GPUs within a corresponding frame period.
 4. The method ofclaim 1, further comprising: dividing responsibility for rendering ofgeometry of graphics for the application between the plurality of GPUsbased on the plurality of screen regions, each of the plurality of GPUshaving a corresponding division of the responsibility; and rendering bythe plurality of GPUs each of the plurality of pieces of geometry forthe second image frame using the information regarding each of theplurality of pieces of geometry and its relation to each of theplurality of screen regions.
 5. The method of claim 1, furthercomprising: determining based on the statistics that a first GPU hasfinished rendering corresponding first pieces of geometry of the firstimage frame before a second GPU has finished rendering correspondingsecond pieces of geometry of the first image frame; and assigning moreof the plurality of pieces of geometry to the first GPU than the secondGPU for the geometry testing of the second image frame.
 6. The method ofclaim 1, further comprising: monitoring performance of each of theplurality of GPUs while performing the geometry testing of the secondimage frame; and dynamically reassigning the plurality of pieces ofgeometry to the plurality of GPUs while performing the geometry testingof the second image frame based on the monitoring of the performance ofthe each of the plurality of GPUs.
 7. The method of claim 1, furthercomprising: redividing the responsibility for the rendering of theplurality of pieces of geometry for the second image frame based on thestatistics so that at least one of the corresponding division ofresponsibility for a corresponding GPU is modified.
 8. A non-transitorycomputer-readable medium for performing a method, the computer-readablemedium comprising: program instructions for rendering a first imageframe generated by an application using a plurality of graphicsprocessing units (GPUs); program instructions for generating statisticsfor the rendering of the first image frame; program instructions forassigning a plurality of pieces of geometry of a second image framegenerated by the application to the plurality of GPUs for geometrytesting based on the statistics, wherein the second image frame followsthe first image frame; and program instructions for performing thegeometry testing at the plurality of GPUs on the plurality of pieces ofgeometry to generate information regarding each of the plurality ofpieces of geometry and its relation to each of a plurality of screenregions, the geometry testing performed at each of the plurality of GPUsbased on the assigning.
 9. The non-transitory computer-readable mediumof claim 8, further comprising: program instructions for generatingstatistics for rendering a plurality of previous image frames, whereineach of the plurality of previous image frames is rendered before thesecond image frame is rendered, wherein the plurality of pieces ofgeometry of the second image frame is assigned to the plurality of GPUsfor geometry testing based on the statistics for the plurality ofprevious image fames that includes the statistics for the rendering ofthe first image frame.
 10. The non-transitory computer-readable mediumof claim 9, wherein in the method the statistics for rendering theplurality of previous image frames include timing of rendering geometryfor each of the plurality of image frames by the plurality of GPUswithin a corresponding frame period.
 11. The non-transitorycomputer-readable medium of claim 8, further comprising: programinstructions for dividing responsibility for rendering of geometry ofgraphics for the application between the plurality of GPUs based on theplurality of screen regions, each of the plurality of GPUs having acorresponding division of the responsibility; and program instructionsfor rendering by the plurality of GPUs each of the plurality of piecesof geometry for the second image frame using the information regardingeach of the plurality of pieces of geometry and its relation to each ofthe plurality of screen regions.
 12. The non-transitorycomputer-readable medium of claim 8, further comprising: programinstructions for determining based on the statistics that a first GPUhas finished rendering corresponding first pieces of geometry of thefirst image frame before a second GPU has finished renderingcorresponding second pieces of geometry of the first image frame; andprogram instructions for assigning more of the plurality of pieces ofgeometry to the first GPU than the second GPU for the geometry testingof the second image frame.
 13. The non-transitory computer-readablemedium of claim 8, further comprising: program instructions formonitoring performance of each of the plurality of GPUs while performingthe geometry testing of the second image frame; and program instructionsfor dynamically reassigning the plurality of pieces of geometry to theplurality of GPUs while performing the geometry testing of the secondimage frame based on the monitoring of the performance of the each ofthe plurality of GPUs.
 14. The non-transitory computer-readable mediumof claim 8, further comprising: program instructions for redividing theresponsibility for the rendering of the plurality of pieces of geometryfor the second image frame based on the statistics so that at least oneof the corresponding division of responsibility for a corresponding GPUis modified.
 15. A computer system comprising: a processor; memorycoupled to the processor and having stored therein instructions that, ifexecuted by the computer system, cause the computer system to execute amethod, comprising: rendering a first image frame generated by anapplication using a plurality of graphics processing units (GPUs);generating statistics for the rendering of the first image frame;assigning a plurality of pieces of geometry of a second image framegenerated by the application to the plurality of GPUs for geometrytesting based on the statistics, wherein the second image frame followsthe first image frame; and performing the geometry testing at theplurality of GPUs on the plurality of pieces of geometry to generateinformation regarding each of the plurality of pieces of geometry andits relation to each of a plurality of screen regions, the geometrytesting performed at each of the plurality of GPUs based on theassigning.
 16. The computer system of claim 15, the method furthercomprising: generating statistics for rendering a plurality of previousimage frames, wherein each of the plurality of previous image frames isrendered before the second image frame is rendered, wherein theplurality of pieces of geometry of the second image frame is assigned tothe plurality of GPUs for geometry testing based on the statistics forthe plurality of previous image fames that includes the statistics forthe rendering of the first image frame.
 17. The computer system of claim16, wherein in the method the statistics for rendering the plurality ofprevious image frames include timing of rendering geometry for each ofthe plurality of image frames by the plurality of GPUs within acorresponding frame period.
 18. The method of claim 15, furthercomprising: dividing responsibility for rendering of geometry ofgraphics for the application between the plurality of GPUs based on theplurality of screen regions, each of the plurality of GPUs having acorresponding division of the responsibility; and rendering by theplurality of GPUs each of the plurality of pieces of geometry for thesecond image frame using the information regarding each of the pluralityof pieces of geometry and its relation to each of the plurality ofscreen regions.
 19. The computer system of claim 15, the method furthercomprising: determining based on the statistics that a first GPU hasfinished rendering corresponding first pieces of geometry of the firstimage frame before a second GPU has finished rendering correspondingsecond pieces of geometry of the first image frame; and assigning moreof the plurality of pieces of geometry to the first GPU than the secondGPU for the geometry testing of the second image frame.
 20. The computersystem of claim 15, the method further comprising: monitoringperformance of each of the plurality of GPUs while performing thegeometry testing of the second image frame; and dynamically reassigningthe plurality of pieces of geometry to the plurality of GPUs whileperforming the geometry testing of the second image frame based on themonitoring of the performance of the each of the plurality of GPUs.